What is a common mistake with Factory pattern in Design Patterns?

Using factories for trivial construction adds indirection without solving a real variability problem.