Echoing Lockhart's Mathematician's Lament, he made the point that it is valuable to give kids problems that cannot be solved in a single sitting. That these are, in some sense, the problems that are most valuable, yet never encountered in typical schooling.
He gave the example of the following classic problem he encountered as a ten year old, to which he credits his lifelong love of mathematics.
You are given twelve coins, at most of one of which is defective (either heavier or lighter than others). You are given a weighing balance. Can you determine with exactly three weighings, which, if any, of the coins is defective?
Prof. Berlekamp also talked about his involvement with the Berkeley Math Circle for kids. There are apparently similar math circles for kids in many cities/places across the country. (Here, for instance, is one in Los Angeles: http://www.math.ucla.edu/~radko/circles/ ).