Training compute for machine learning systems has doubled roughly every 6 months. #gpt3 was introduced in 2020 with 170 billion parameters. If the trend continues, we could see models with (almost) 700 trillion parameters, 7 times more than the human brain, by 2030.

The cost of computation goes down exponentially, roughly two orders of magnitude per decade. From 1940 to 2020, the amount of available computing power per constant dollar has increased by 16 orders of magnitude. #computation #AI #technology