“Surely a Network That’s Big Enough Can Do Anything!”

There are things that can be figured out by formal processes, but aren’t readily accessible to immediate human thinking.

  • Computational irreducibility means that we can never guarantee that the unexpected won’t happen-and it’s only by explicitly doing the computation that you can tell what actually happens in any particular case.
  • There’s an ultimate tradeoff between capability and trainability: the more you want a system to make “true use” of its computational capabilities, the more it’s going to show computational irreduciibility, and the less it is going to be trainable.

Sign Up for NextBigWhat Newsletter

Curated. Summarized. Important News. For free.