So why can't I just have a rulebook a handful of codexes and my models to play a game. People tell me I need apocalypse and escalation and dataslates and psychic cards.
You
can still run with just Core and Codex, depending on who you're playing with. Granted, GW stores tend to throw open the floodgates, as they're trying to push as much product as they can. Playing with everything can get a bit overwhelming at times.
In short, the main things they added are Escalation, to add Apocalypse sized Super-Heavies into "standard" games of 40k. These are generally single unit slots called "Lords of War", and so you can bring a Stompa or Baneblade (or many other options now) into ... say, a 1500 point game. There's a few extra rules there (ie: you get extra Victory Points for causing damage to Super-Heavies) also tossed in there. While it's neat to dust off that old over-sized Warhound every now and again, having such a large model (that takes up half your point allotment) ends up being the focal point of the game. And that can get old after the billionth time.
Second item that's often included is Stronghold Assault, which are the rules for Fortifications (ie: buildings) to be added to your army. There's one or two broken items in the book that people cherry pick to add to their army. Someone else can probably go into more detail on it. We tend to avoid it, because the old "have fun storming the castle" line gets old pretty quick amongst my group.
Then you have all the dataslates/supplements/formations/etc. These are basically units pulled from books and given slightly special rules. I'm rather condensing a lot of various books into this one category, because they all behave in a similar fashion. Most do not take up your allies slot while some, such as army supplements like the Black Legion, do. They all have little tweaks to the rules, by either having their own Warlord/Relic chart, or changes to the rules, like having Tyrannic War Vets having Preferred Enemey (Tyranids). If you remember the old slew of Chapter Approved releases from the late 3rd ed, this is much like the modern version of it.
The main way this has changed 40k is that while it's provided a lot of variety to lists, it's near impossible to keep up with everything that's out there. At least, without going broke. As a customer, you rather have to narrow down your focus, and just buy/collect that. If you're one of those people that has to know every rule out there so they know what they can potentially face, and won't be "blindsided" by unexpected rules when they do, then you'll absolutely hate (with an avid passion) the plethora of releases we're seeing today. If you're part of a gaming group that has no problem self-regulating, then it's a great time to be in the hobby.
This also means that while the "gaming group with buddies" crowd is having a good old time, the "random pickup games with strangers" crowd is really struggling to make an enjoyable experience of it.
As an aside, you're right. This sticky has outlived it's usefulness.