Key Concepts and Formulas

• Parallel Vectors from Cross Product: If $\vec{X} \times \vec{Y} = \vec{0}$ for non-zero vectors $\vec{X}$ and $\vec{Y}$, then $\vec{X}$ and $\vec{Y}$ are parallel, meaning $\vec{X} = k\vec{Y}$ for some scalar $k$.
• Distributive Property of Cross Product: $\vec{A} \times (\vec{B} + \vec{C}) = \vec{A} \times \vec{B} + \vec{A} \times \vec{C}$.
• Cross Product of a Vector with Itself: $\vec{V} \times \vec{V} = \vec{0}$.
• Dot Product: For $\vec{U} = U_x\hat{i} + U_y\hat{j} + U_z\hat{k}$ and $\vec{V} = V_x\hat{i} + V_y\hat{j} + V_z\hat{k}$, $\vec{U} \cdot \vec{V} = U_x V_x + U_y V_y + U_z V_z$.
• Magnitude Squared of a Vector: For $\vec{V} = V_x\hat{i} + V_y\hat{j} + V_z\hat{k}$, $|\vec{V}|^2 = V_x^2 + V_y^2 + V_z^2$.

Step-by-Step Solution

Step 1: Simplify the cross product condition. We are given $(\vec{a}+\vec{b}+\vec{c}) \times \vec{c}=\overrightarrow{0}$. Using the distributive property of the cross product, we can rewrite this as $(\vec{a}+\vec{b}) \times \vec{c} + \vec{c} \times \vec{c} = \overrightarrow{0}$. Since $\vec{c} \times \vec{c} = \overrightarrow{0}$, the equation simplifies to $(\vec{a}+\vec{b}) \times \vec{c} = \overrightarrow{0}$. This implies that the vector $(\vec{a}+\vec{b})$ and vector $\vec{c}$ are parallel. Therefore, $\vec{c}$ can be expressed as a scalar multiple of $(\vec{a}+\vec{b})$. Let $\vec{c} = \alpha (\vec{a} + \vec{b})$ for some scalar $\alpha$.

Step 2: Calculate $\vec{a} + \vec{b}$. Given $\vec{a} = \lambda \hat{i} + \hat{j} - \hat{k}$ and $\vec{b} = 3\hat{i} - \hat{j} + 2\hat{k}$, their sum is: $\vec{a} + \vec{b} = (\lambda \hat{i} + \hat{j} - \hat{k}) + (3\hat{i} - \hat{j} + 2\hat{k})$ $\vec{a} + \vec{b} = (\lambda + 3)\hat{i} + (1-1)\hat{j} + (-1+2)\hat{k}$ $\vec{a} + \vec{b} = (\lambda + 3)\hat{i} + \hat{k}$

Step 3: Express $\vec{c}$ in terms of $\alpha$ and $\lambda$. Using the result from Step 1 and Step 2: $\vec{c} = \alpha ((\lambda + 3)\hat{i} + \hat{k})$ $\vec{c} = \alpha(\lambda + 3)\hat{i} + 0\hat{j} + \alpha\hat{k}$

Step 4: Use the dot product conditions to form algebraic equations. We are given $\vec{a} \cdot \vec{c} = -17$ and $\vec{b} \cdot \vec{c} = -20$.

For $\vec{a} \cdot \vec{c} = -17$: $(\lambda \hat{i} + \hat{j} - \hat{k}) \cdot (\alpha(\lambda + 3)\hat{i} + 0\hat{j} + \alpha\hat{k}) = -17$ $\lambda(\alpha(\lambda + 3)) + (1)(0) + (-1)(\alpha) = -17$ $\alpha\lambda(\lambda + 3) - \alpha = -17$ $\alpha(\lambda^2 + 3\lambda - 1) = -17 \quad (*)$

For $\vec{b} \cdot \vec{c} = -20$: $(3\hat{i} - \hat{j} + 2\hat{k}) \cdot (\alpha(\lambda + 3)\hat{i} + 0\hat{j} + \alpha\hat{k}) = -20$ $(3)(\alpha(\lambda + 3)) + (-1)(0) + (2)(\alpha) = -20$ $3\alpha(\lambda + 3) + 2\alpha = -20$ $3\alpha\lambda + 9\alpha + 2\alpha = -20$ $\alpha(3\lambda + 11) = -20 \quad (**)$

Step 5: Solve the system of equations for $\alpha$ and $\lambda$. From equation $(**)$, assuming $3\lambda + 11 \neq 0$, we get $\alpha = \frac{-20}{3\lambda + 11}$. Substitute this into equation $(*)$: $\left(\frac{-20}{3\lambda + 11}\right)(\lambda^2 + 3\lambda - 1) = -17$ Multiply both sides by $(3\lambda + 11)$ and by $-1$: $20(\lambda^2 + 3\lambda - 1) = 17(3\lambda + 11)$ $20\lambda^2 + 60\lambda - 20 = 51\lambda + 187$ Rearrange into a quadratic equation: $20\lambda^2 + (60-51)\lambda + (-20-187) = 0$ $20\lambda^2 + 9\lambda - 207 = 0$ Using the quadratic formula $\lambda = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a}$: $\lambda = \frac{-9 \pm \sqrt{9^2 - 4(20)(-207)}}{2(20)}$ $\lambda = \frac{-9 \pm \sqrt{81 + 16560}}{40}$ $\lambda = \frac{-9 \pm \sqrt{16641}}{40}$ We find that $\sqrt{16641} = 129$. $\lambda = \frac{-9 \pm 129}{40}$ This gives two possible values for $\lambda$: $\lambda_1 = \frac{-9 + 129}{40} = \frac{120}{40} = 3$ $\lambda_2 = \frac{-9 - 129}{40} = \frac{-138}{40} = -\frac{69}{20}$ Since $\lambda \in \mathbb{Z}$, we must have $\lambda = 3$.

Now, substitute $\lambda = 3$ back into the expression for $\alpha$: $\alpha = \frac{-20}{3(3) + 11} = \frac{-20}{9 + 11} = \frac{-20}{20} = -1$ So, $\lambda = 3$ and $\alpha = -1$.

Step 6: Determine the vector $\vec{c}$. Using $\vec{c} = \alpha((\lambda + 3)\hat{i} + \hat{k})$ with $\alpha = -1$ and $\lambda = 3$: $\vec{c} = -1((3 + 3)\hat{i} + \hat{k})$ $\vec{c} = -1(6\hat{i} + \hat{k})$ $\vec{c} = -6\hat{i} - \hat{k}$

Step 7: Calculate the vector for the cross product. Let $\vec{D} = \lambda \hat{i} + \hat{j} + \hat{k}$. Substitute $\lambda = 3$: $\vec{D} = 3\hat{i} + \hat{j} + \hat{k}$

Step 8: Compute the cross product $\vec{c} \times \vec{D}$. $\vec{c} \times \vec{D} = (-6\hat{i} - \hat{k}) \times (3\hat{i} + \hat{j} + \hat{k})$ Using the determinant form: $\vec{c} \times \vec{D} = \begin{vmatrix} \hat{i} & \hat{j} & \hat{k} \\ -6 & 0 & -1 \\ 3 & 1 & 1 \end{vmatrix}$ $= \hat{i}((0)(1) - (-1)(1)) - \hat{j}((-6)(1) - (-1)(3)) + \hat{k}((-6)(1) - (0)(3))$ $= \hat{i}(0 + 1) - \hat{j}(-6 + 3) + \hat{k}(-6 - 0)$ $= \hat{i}(1) - \hat{j}(-3) + \hat{k}(-6)$ $\vec{c} \times \vec{D} = \hat{i} + 3\hat{j} - 6\hat{k}$

Step 9: Calculate the magnitude squared of the resulting vector. $|\vec{c} \times \vec{D}|^2 = | \hat{i} + 3\hat{j} - 6\hat{k} |^2$ $|\vec{c} \times \vec{D}|^2 = (1)^2 + (3)^2 + (-6)^2$ $|\vec{c} \times \vec{D}|^2 = 1 + 9 + 36$ $|\vec{c} \times \vec{D}|^2 = 46$

Common Mistakes & Tips

• Algebraic Errors: Solving the quadratic equation for $\lambda$ or simplifying the algebraic expressions can lead to errors. Double-check all calculations.
• Forgetting the Integer Constraint: Always remember to use the condition that $\lambda$ is an integer to select the correct value of $\lambda$.
• Mistaking Dot and Cross Products: Ensure you are applying the correct vector operation for each condition.

Summary The problem leverages the property that if the cross product of two vectors is zero, they are parallel. This allowed us to express $\vec{c}$ as a scalar multiple of $\vec{a} + \vec{b}$. By substituting this into the given dot product conditions, we formed a system of scalar equations. Solving this system, and utilizing the constraint that $\lambda$ is an integer, allowed us to find the values of $\lambda$ and the scalar multiplier $\alpha$. Finally, we determined the vector $\vec{c}$ and computed the magnitude squared of the required cross product.

The final answer is $\boxed{46}$.