Key Concepts and Formulas

• Coplanarity of Vectors: Three vectors $\vec{a}$, $\vec{b}$, and $\vec{c}$ are coplanar if their scalar triple product is zero, i.e., $[\vec{a}, \vec{b}, \vec{c}] = \vec{a} \cdot (\vec{b} \times \vec{c}) = 0$. Alternatively, if $\vec{a}$ and $\vec{b}$ are non-collinear, any vector $\vec{c}$ coplanar with them can be expressed as a linear combination: $\vec{c} = x\vec{a} + y\vec{b}$, where $x$ and $y$ are scalars.
• Perpendicular Vectors: Two vectors are perpendicular if their dot product is zero. For example, if $\vec{u}$ is perpendicular to $\vec{v}$, then $\vec{u} \cdot \vec{v} = 0$.
• Dot Product: The dot product of two vectors $\vec{u} = u_1 \hat{i} + u_2 \hat{j} + u_3 \hat{k}$ and $\vec{v} = v_1 \hat{i} + v_2 \hat{j} + v_3 \hat{k}$ is given by $\vec{u} \cdot \vec{v} = u_1 v_1 + u_2 v_2 + u_3 v_3$.
• Magnitude of a Vector: The magnitude of a vector $\vec{u} = u_1 \hat{i} + u_2 \hat{j} + u_3 \hat{k}$ is given by $|\vec{u}| = \sqrt{u_1^2 + u_2^2 + u_3^2}$.

Step-by-Step Solution

Step 1: Represent the unknown vector $\vec{c}$ using the coplanarity condition. Given that $\vec{c}$ is coplanar with $\vec{a}$ and $\vec{b}$, and $\vec{a}$ and $\vec{b}$ are not collinear (as their components are not proportional), we can express $\vec{c}$ as a linear combination of $\vec{a}$ and $\vec{b}$. Let $\vec{c} = x\vec{a} + y\vec{b}$, where $x$ and $y$ are scalar constants. Substituting the given vectors: $\vec{c} = x(\hat{i} + 2\hat{j} + 3\hat{k}) + y(3\hat{i} + \hat{j} - \hat{k})$ $\vec{c} = (x + 3y)\hat{i} + (2x + y)\hat{j} + (3x - y)\hat{k}$

Step 2: Use the perpendicularity condition to form an equation involving $x$ and $y$. We are given that $\vec{c}$ is perpendicular to $\vec{b}$. This means their dot product is zero: $\vec{c} \cdot \vec{b} = 0$. We have $\vec{b} = 3\hat{i} + \hat{j} - \hat{k}$. So, $\vec{c} \cdot \vec{b} = [(x + 3y)\hat{i} + (2x + y)\hat{j} + (3x - y)\hat{k}] \cdot (3\hat{i} + \hat{j} - \hat{k}) = 0$. Calculating the dot product: $3(x + 3y) + 1(2x + y) - 1(3x - y) = 0$ $3x + 9y + 2x + y - 3x + y = 0$ Combine like terms: $(3x + 2x - 3x) + (9y + y + y) = 0$ $2x + 11y = 0$ From this equation, we can express one variable in terms of the other. Let's express $x$ in terms of $y$: $2x = -11y \implies x = -\frac{11}{2}y$

Step 3: Use the dot product condition $\vec{a} \cdot \vec{c} = 5$ to find the specific values of $x$ and $y$. We are given $\vec{a} \cdot \vec{c} = 5$. We have $\vec{a} = \hat{i} + 2\hat{j} + 3\hat{k}$. So, $\vec{a} \cdot \vec{c} = (\hat{i} + 2\hat{j} + 3\hat{k}) \cdot [(x + 3y)\hat{i} + (2x + y)\hat{j} + (3x - y)\hat{k}] = 5$. Calculating the dot product: $1(x + 3y) + 2(2x + y) + 3(3x - y) = 5$ $x + 3y + 4x + 2y + 9x - 3y = 5$ Combine like terms: $(x + 4x + 9x) + (3y + 2y - 3y) = 5$ $14x + 2y = 5$

Step 4: Substitute the relationship between $x$ and $y$ from Step 2 into the equation from Step 3. We found $x = -\frac{11}{2}y$. Substitute this into $14x + 2y = 5$: $14\left(-\frac{11}{2}y\right) + 2y = 5$ $7(-11y) + 2y = 5$ $-77y + 2y = 5$ $-75y = 5$ $y = -\frac{5}{75} = -\frac{1}{15}$

Step 5: Calculate the value of $x$ using the value of $y$. Using $x = -\frac{11}{2}y$: $x = -\frac{11}{2}\left(-\frac{1}{15}\right)$ $x = \frac{11}{30}$

Step 6: Find the components of vector $\vec{c}$ using the values of $x$ and $y$. Recall $\vec{c} = (x + 3y)\hat{i} + (2x + y)\hat{j} + (3x - y)\hat{k}$. Substitute $x = \frac{11}{30}$ and $y = -\frac{1}{15}$: $x + 3y = \frac{11}{30} + 3\left(-\frac{1}{15}\right) = \frac{11}{30} - \frac{3}{15} = \frac{11}{30} - \frac{6}{30} = \frac{5}{30} = \frac{1}{6}$ $2x + y = 2\left(\frac{11}{30}\right) + \left(-\frac{1}{15}\right) = \frac{22}{30} - \frac{1}{15} = \frac{11}{15} - \frac{1}{15} = \frac{10}{15} = \frac{2}{3}$ $3x - y = 3\left(\frac{11}{30}\right) - \left(-\frac{1}{15}\right) = \frac{33}{30} + \frac{1}{15} = \frac{11}{10} + \frac{1}{15} = \frac{33}{30} + \frac{2}{30} = \frac{35}{30} = \frac{7}{6}$ So, $\vec{c} = \frac{1}{6}\hat{i} + \frac{2}{3}\hat{j} + \frac{7}{6}\hat{k}$.

Step 7: Calculate the magnitude of vector $\vec{c}$. $|\vec{c}| = \sqrt{\left(\frac{1}{6}\right)^2 + \left(\frac{2}{3}\right)^2 + \left(\frac{7}{6}\right)^2}$ $|\vec{c}| = \sqrt{\frac{1}{36} + \frac{4}{9} + \frac{49}{36}}$ To add these fractions, find a common denominator, which is 36. $\frac{4}{9} = \frac{4 \times 4}{9 \times 4} = \frac{16}{36}$ So, $|\vec{c}| = \sqrt{\frac{1}{36} + \frac{16}{36} + \frac{49}{36}}$ $|\vec{c}| = \sqrt{\frac{1 + 16 + 49}{36}}$ $|\vec{c}| = \sqrt{\frac{66}{36}}$ Simplify the fraction: $|\vec{c}| = \sqrt{\frac{11 \times 6}{6 \times 6}} = \sqrt{\frac{11}{6}}$

Common Mistakes & Tips

• Algebraic Errors: Carefully perform algebraic manipulations, especially when dealing with fractions and multiple variables. Double-check each step to avoid calculation mistakes.
• Misinterpreting Coplanarity: Remember that if $\vec{a}$ and $\vec{b}$ are non-collinear, coplanarity implies that $\vec{c}$ can be written as a linear combination of $\vec{a}$ and $\vec{b}$. If $\vec{a}$ and $\vec{b}$ were collinear, the representation would be different.
• Order of Operations: Be meticulous with the order of operations when calculating dot products and magnitudes.

Summary

The problem is solved by first representing the unknown vector $\vec{c}$ as a linear combination of the two given coplanar vectors $\vec{a}$ and $\vec{b}$. The conditions of perpendicularity ($\vec{c} \cdot \vec{b} = 0$) and the given dot product ($\vec{a} \cdot \vec{c} = 5$) are then used to form a system of linear equations for the scalar coefficients of the linear combination. Solving this system yields the explicit form of $\vec{c}$, after which its magnitude is calculated.

The final answer is $\boxed{\sqrt{\frac{11}{6}}}$, which corresponds to option (A).