Key Concepts and Formulas

• Orthocenter: The point of intersection of the altitudes of a triangle.
• Slope of a line: Given two points $(x_1, y_1)$ and $(x_2, y_2)$, the slope $m = \frac{y_2 - y_1}{x_2 - x_1}$.
• Perpendicular lines: If two lines are perpendicular, the product of their slopes is -1 (i.e., $m_1 \cdot m_2 = -1$).
• Equation of a circle: The equation of a circle with center $(a, b)$ and radius $r$ is $(x - a)^2 + (y - b)^2 = r^2$.

Step-by-Step Solution

Step 1: Find the slope of AB.

The slope of the line segment AB, where A(5, -2) and B(8, 3), is given by: $m_{AB} = \frac{3 - (-2)}{8 - 5} = \frac{5}{3}$

Step 2: Find the slope of CP.

Since CP is perpendicular to AB, the product of their slopes is -1. Therefore, the slope of CP is: $m_{CP} = -\frac{1}{m_{AB}} = -\frac{1}{\frac{5}{3}} = -\frac{3}{5}$

Step 3: Express the slope of CP in terms of h and k.

The slope of CP, where C(h, k) and P(6, 1), is also given by: $m_{CP} = \frac{k - 1}{h - 6}$

Step 4: Equate the two expressions for the slope of CP and simplify.

We now have two expressions for $m_{CP}$, so we can equate them: $\frac{k - 1}{h - 6} = -\frac{3}{5}$ Cross-multiplying, we get: $5(k - 1) = -3(h - 6)$ $5k - 5 = -3h + 18$ $3h + 5k = 23 \hspace{1cm} (1)$

Step 5: Find the slope of BC.

The slope of the line segment BC, where B(8, 3) and C(h, k), is given by: $m_{BC} = \frac{k - 3}{h - 8}$

Step 6: Find the slope of AP.

The slope of the line segment AP, where A(5, -2) and P(6, 1), is given by: $m_{AP} = \frac{1 - (-2)}{6 - 5} = \frac{3}{1} = 3$

Step 7: Use the perpendicularity of AP and BC.

Since AP is perpendicular to BC, the product of their slopes is -1. Therefore: $m_{AP} \cdot m_{BC} = -1$ $3 \cdot \frac{k - 3}{h - 8} = -1$ $3(k - 3) = -(h - 8)$ $3k - 9 = -h + 8$ $h + 3k = 17 \hspace{1cm} (2)$

Step 8: Solve the system of linear equations (1) and (2) for h and k.

We have the following system of equations: $3h + 5k = 23 \hspace{1cm} (1)$ $h + 3k = 17 \hspace{1cm} (2)$ Multiply equation (2) by 3: $3h + 9k = 51 \hspace{1cm} (3)$ Subtract equation (1) from equation (3): $(3h + 9k) - (3h + 5k) = 51 - 23$ $4k = 28$ $k = 7$ Substitute k = 7 into equation (2): $h + 3(7) = 17$ $h + 21 = 17$ $h = -4$ Thus, C is the point (-4, 7).

Step 9: Determine which circle the point C lies on.

We need to check which of the given circle equations is satisfied by the point (-4, 7). (A) $x^2 + y^2 - 74 = 0$: $(-4)^2 + (7)^2 - 74 = 16 + 49 - 74 = 65 - 74 = -9 \neq 0$ (B) $x^2 + y^2 - 65 = 0$: $(-4)^2 + (7)^2 - 65 = 16 + 49 - 65 = 65 - 65 = 0$. Therefore, the point C(-4, 7) lies on the circle $x^2 + y^2 - 65 = 0$.

Step 10: Recalculate and verify based on the given answer.

Since the provided answer is A ($x^2+y^2-74=0$), there must be an error in the provided answer. Let's recheck the calculations. The coordinates are A(5,-2), B(8,3), P(6,1), and C(h,k).

$m_{AB} = \frac{3-(-2)}{8-5} = \frac{5}{3}$ $m_{CP} = -\frac{3}{5} = \frac{k-1}{h-6}$. Thus $5k-5 = -3h+18$, so $3h+5k = 23$. $m_{BC} = \frac{k-3}{h-8}$ $m_{AP} = \frac{1-(-2)}{6-5} = 3$ $m_{BC} = -\frac{1}{3} = \frac{k-3}{h-8}$. Thus $h-8 = -3k+9$, so $h+3k = 17$. $3h+5k=23$ and $h+3k=17$. $3h+9k=51$, so $4k=28$ and $k=7$. $h+3(7) = 17$, so $h=17-21 = -4$. C is (-4,7). If C lies on $x^2+y^2-74=0$, then $(-4)^2+7^2=74$, so $16+49=74$, but $65 \neq 74$. The question is incorrect, the correct answer is B ($x^2+y^2-65=0$). If we have to assume that the given answer is correct, we need to find the orthocenter corresponding to option A. Let C(h,k) lie on $x^2+y^2=74$. $h^2+k^2=74$.

Common Mistakes & Tips

• Double-check the signs when calculating slopes and using the perpendicularity condition.
• Be careful when solving the system of equations to avoid algebraic errors.
• Remember to substitute the coordinates of the point into the correct circle equation.

Summary

We used the properties of the orthocenter and the perpendicularity of altitudes to set up a system of two linear equations in terms of the coordinates (h, k) of vertex C. Solving this system, we found the coordinates of C to be (-4, 7). Then, we checked which of the given circle equations is satisfied by this point. We found that the point lies on the circle $x^2 + y^2 - 65 = 0$. Based on the provided answer, the correct answer should be $x^2+y^2-74=0$. However, the point (-4,7) does not satisfy this equation. The correct answer is B. There appears to be an error in the question's provided answer.

Final Answer

The final answer is \boxed{B}, which corresponds to option (B).