Key Concepts and Formulas

• Coplanarity of Vectors: Three vectors $\overrightarrow a$, $\overrightarrow b$, and $\overrightarrow c$ are coplanar if they lie in the same plane. This occurs if and only if their scalar triple product is zero.
• Scalar Triple Product: The scalar triple product of three vectors $\overrightarrow a = a_1\widehat i + a_2\widehat j + a_3\widehat k$, $\overrightarrow b = b_1\widehat i + b_2\widehat j + b_3\widehat k$, and $\overrightarrow c = c_1\widehat i + c_2\widehat j + c_3\widehat k$ is given by the determinant of the matrix formed by their components: $[\overrightarrow a \ \overrightarrow b \ \overrightarrow c] = \begin{vmatrix} a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \\ c_1 & c_2 & c_3 \end{vmatrix}$
• Condition for Coplanarity: For vectors $\overrightarrow a$, $\overrightarrow b$, and $\overrightarrow c$ to be coplanar, their scalar triple product must be zero: $\begin{vmatrix} a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \\ c_1 & c_2 & c_3 \end{vmatrix} = 0$

Step-by-Step Solution

Step 1: Set up the determinant for coplanarity. We are given the vectors $\overrightarrow a = \alpha \widehat i + \widehat j + 3\widehat k$, $\overrightarrow b = 2\widehat i + \widehat j - \alpha \widehat k$, and $\overrightarrow c = \alpha \widehat i - 2\widehat j + 3\widehat k$. For these vectors to be coplanar, their scalar triple product must be zero. We form a determinant using the components of these vectors: $\begin{vmatrix} \alpha & 1 & 3 \\ 2 & 1 & -\alpha \\ \alpha & -2 & 3 \end{vmatrix} = 0$

Step 2: Expand the determinant. We expand the determinant along the first row. The formula for a $3 \times 3$ determinant is $a(ei - fh) - b(di - fg) + c(dh - eg)$. $\alpha \left( (1)(3) - (-\alpha)(-2) \right) - 1 \left( (2)(3) - (-\alpha)(\alpha) \right) + 3 \left( (2)(-2) - (1)(\alpha) \right) = 0$

Step 3: Simplify the expanded expression. Now, we simplify each term within the parentheses and then distribute: $\alpha (3 - 2\alpha) - 1 (6 + \alpha^2) + 3 (-4 - \alpha) = 0$ Distributing the terms: $(3\alpha - 2\alpha^2) - (6 + \alpha^2) + (-12 - 3\alpha) = 0$ $3\alpha - 2\alpha^2 - 6 - \alpha^2 - 12 - 3\alpha = 0$

Step 4: Combine like terms to form a polynomial equation in $\alpha$. We group the terms by powers of $\alpha$: $(-2\alpha^2 - \alpha^2) + (3\alpha - 3\alpha) + (-6 - 12) = 0$ $-3\alpha^2 + 0\alpha - 18 = 0$ $-3\alpha^2 - 18 = 0$

Step 5: Solve the polynomial equation for $\alpha$. We solve the equation $-3\alpha^2 - 18 = 0$ for $\alpha$: $-3\alpha^2 = 18$ Divide both sides by $-3$: $\alpha^2 = \frac{18}{-3}$ $\alpha^2 = -6$

Step 6: Analyze the solution in the context of real numbers. The problem states that $\alpha \in \mathbb{R}$, meaning $\alpha$ must be a real number. For any real number $\alpha$, its square, $\alpha^2$, must be non-negative ($\alpha^2 \ge 0$). The equation $\alpha^2 = -6$ has no real solutions because the square of any real number cannot be negative. Therefore, there are no real values of $\alpha$ for which the given vectors are coplanar.

Step 7: Determine the set S and choose the correct option. The set $S = \{\alpha \in \mathbb{R} : \overrightarrow a, \overrightarrow b, \overrightarrow c \text{ are coplanar}\}$ is the set of all real values of $\alpha$ that satisfy the coplanarity condition. Since we found no such real values, the set $S$ is empty. $S = \emptyset$. This corresponds to option (D).

Common Mistakes & Tips

• Sign Errors in Determinant Expansion: Be meticulous with the signs when calculating determinants. The alternating signs $(+,-,+)$ for expansion along the first row are crucial.
• Algebraic Simplification: Double-check your algebra when distributing and combining terms. Small errors in arithmetic can lead to incorrect polynomial equations.
• Real Number Constraint: Always remember the domain of the variable. If $\alpha$ is restricted to real numbers, then equations like $\alpha^2 = -6$ have no solutions.

Summary

For three vectors to be coplanar, their scalar triple product must be zero. By forming a determinant of the components of the given vectors and setting it to zero, we obtained the equation $\alpha^2 = -6$. Since $\alpha$ is restricted to be a real number, there is no real value of $\alpha$ that satisfies this equation. Consequently, the set of such $\alpha$ values is empty.

The final answer is $\boxed{(D)}$ which corresponds to the set being empty.