1. Key Concepts and Formulas

• Definition of a function as a maximum of two functions: $f(x) = \max\{g(x), h(x)\}$ means that for any given $x$, $f(x)$ takes the value of whichever of $g(x)$ or $h(x)$ is larger.
• Differentiability at a point: A function $f(x)$ is differentiable at a point $x=a$ if the limit of the difference quotient exists: $\lim_{h \to 0} \frac{f(a+h) - f(a)}{h}$ Geometrically, this means the graph of the function has a well-defined tangent line at that point. Functions are not differentiable at sharp corners, cusps, or points of discontinuity.
• Finding points of non-differentiability for $\max\{g(x), h(x)\}$: Points where $f(x) = \max\{g(x), h(x)\}$ might not be differentiable are typically where $g(x) = h(x)$ and the derivatives of $g(x)$ and $h(x)$ are different, leading to a "corner" in the graph of $f(x)$.

2. Step-by-Step Solution

Step 1: Define the function $f(x)$ piecewise. We are given $f(x) = \max\{x, x^2\}$. To understand where the function might change its behavior, we need to find the points where $x = x^2$. $x = x^2$ $x^2 - x = 0$ $x(x - 1) = 0$ This equation holds for $x=0$ and $x=1$. These are the critical points where the definition of $f(x)$ might switch between $x$ and $x^2$.

Now, let's analyze the intervals defined by these points:

• Interval 1: $x < 0$ In this interval, $x^2 > 0$ and $x < 0$. Thus, $x^2 > x$. So, $f(x) = x^2$ for $x < 0$.
• Interval 2: $0 \le x < 1$ In this interval, consider a value like $x = 0.5$. $x = 0.5$ and $x^2 = 0.25$. So $x > x^2$. Thus, $f(x) = x$ for $0 \le x < 1$.
• Interval 3: $x \ge 1$ In this interval, consider a value like $x = 2$. $x = 2$ and $x^2 = 4$. So $x^2 > x$. Thus, $f(x) = x^2$ for $x \ge 1$.

Therefore, the piecewise definition of $f(x)$ is: $f(x) = \begin{cases} x^2 & \text{if } x < 0 \\ x & \text{if } 0 \le x < 1 \\ x^2 & \text{if } x \ge 1 \end{cases}$

Step 2: Check for differentiability at the critical points $x=0$ and $x=1$. A function is not differentiable at points where its graph has a sharp corner. This often occurs at the points where the definition of the function changes, especially if the derivatives of the constituent functions are different at those points.

Case 1: Check differentiability at $x=0$. For $x < 0$, $f(x) = x^2$, so $f'(x) = 2x$. The left-hand derivative at $x=0$ is: $f'_{-}(0) = \lim_{x \to 0^-} f'(x) = \lim_{x \to 0^-} 2x = 2(0) = 0$ For $0 \le x < 1$, $f(x) = x$, so $f'(x) = 1$. The right-hand derivative at $x=0$ is: $f'_{+}(0) = \lim_{x \to 0^+} f'(x) = \lim_{x \to 0^+} 1 = 1$ Since the left-hand derivative ($0$) is not equal to the right-hand derivative ($1$) at $x=0$, the function $f(x)$ is not differentiable at $x=0$.

Case 2: Check differentiability at $x=1$. For $0 \le x < 1$, $f(x) = x$, so $f'(x) = 1$. The left-hand derivative at $x=1$ is: $f'_{-}(1) = \lim_{x \to 1^-} f'(x) = \lim_{x \to 1^-} 1 = 1$ For $x \ge 1$, $f(x) = x^2$, so $f'(x) = 2x$. The right-hand derivative at $x=1$ is: $f'_{+}(1) = \lim_{x \to 1^+} f'(x) = \lim_{x \to 1^+} 2x = 2(1) = 2$ Since the left-hand derivative ($1$) is not equal to the right-hand derivative ($2$) at $x=1$, the function $f(x)$ is not differentiable at $x=1$.

Step 3: Identify the set S. The set $S$ consists of all points where $f$ is not differentiable. From our analysis in Step 2, we found that $f$ is not differentiable at $x=0$ and $x=1$. Therefore, $S = \{0, 1\}$.

3. Common Mistakes & Tips

• Assuming differentiability where functions meet: Do not assume that a function defined as the maximum (or minimum) of two other functions is differentiable at every point where the two functions are equal. Always check the derivatives from both sides.
• Ignoring endpoints of intervals: When defining piecewise functions, pay close attention to the inequality signs ($\le$, $<$) as they determine which function applies at the exact point.
• Graphical intuition is helpful but not sufficient: While sketching the graph can quickly suggest points of non-differentiability (sharp corners), a rigorous check using limits of derivatives is necessary for a definitive answer.

4. Summary

The function $f(x) = \max\{x, x^2\}$ was analyzed by first determining its piecewise definition based on where $x$ and $x^2$ are equal. The critical points where the definition might change are $x=0$ and $x=1$. We then examined the differentiability of $f(x)$ at these points by comparing the left-hand and right-hand derivatives. At $x=0$, the left-hand derivative of $f(x)$ (approaching from $x^2$) is 0, and the right-hand derivative (approaching from $x$) is 1. Since these are unequal, $f(x)$ is not differentiable at $x=0$. Similarly, at $x=1$, the left-hand derivative (approaching from $x$) is 1, and the right-hand derivative (approaching from $x^2$) is 2. Since these are unequal, $f(x)$ is not differentiable at $x=1$. Thus, the set $S$ of points where $f$ is not differentiable is $\{0, 1\}$.

5. Final Answer

The final answer is \boxed{{0, 1}} which corresponds to option (A).