CS 464: Introduction to Computer Graphics

This code computes a normal to the plane made by any three points

//  pt1, pt2, and pt3 are vec3 objects.
// for example:
//   pt1 = [x1,y1,z1];  pt2 = [x2,y2,z2], pt3 = [x3,y3,z3];
function normalTri(pt1,pt2,tp3)
{
     // to make the math easier first we reverse our common point
     vec3.negate(pt1,negpt1);
     // now calculate two vectors for the plane
     vec3.add(pt2,pt1,vecpt2);  // calculate vector pt1-->pt2
     vec3.add(pt3,pt1,vecpt3);  // cacluate vector pt1-->pt3

     // now if we take the cross product that gives us the normal
     vec3.cross(vecpt2,vecpt3,normal);
     return normal;
}

Remember every plane has two normals that point in opposite directions. You might need to come up with a way to check the resultant normal to make certain it is pointing in the positive Y direction or your screen images might flip up and down.