Difference between C and CE on calculators
According to Vintage Technology, both buttons are a way to clear or cancel an entry. The C button will clear all input to the calculator. The CE button clears the most recent entry, so if you make a mistake in a long computation, you don't need to start all over again.
Exactly when calculators began to use these buttons is hard to say. The Vintage Calculators Web Museum provides a timeline that seems to show that the Friden EC-130, which was introduced in 1964, included "clear entry" and "clear display" keys (as did its successor, the EC-132). Those may not be the exact C and CE keys we see today, but it sounds like they performed the same function.