This is a very thoughtful question, thank-you. The book -- which was supposed to be out ten years' ago -- is based on a wide range of factors but the trajectory began more than two decades ago. My original major in college was exercise physiology, and my work was in helping athletes work at their highest levels of performance. From there, it was a step into helping NON-pro athletes how to develop high levels of fitness and ultimately skill building. I was simultaneously working on my own skill building as a professional skateboarder in the late 70's and early 80's.
But then I discovered the field of artificial intelligence (as it was in the 80's) and that set me on a permanent path into computer science. I went back to take some AI and compsci courses at UCLA, and eventually taught interaction design courses there. The goal then was to do 'knowledge engineering' so that we could represent in software algorithms (then, mostly rule-based systems) the ways in which an "expert" performed. Of course we all learned the hard and painful way that this was a flawed idea from the very beginning for most forms of deep expertise, although these rule-based AI systems were particularly useful for configuration and planning, the area that Bert Bates specialized in back in HIS artificial intelligence days
Then I worked in game development and design for a long time but my love was always around helping people build knowledge and skills in the most effective ways, something that was the initial inspiration for javaranch back in 1997, eventually leading to Head First Java, and now today is reflected best by the concepts in Badass.
This was a painful process though, as so much of learning theory is either completely ignored or flat-out wrong. Almost all of us are far more familiar with traditional ways of teaching, learning, classrooms, books, etc. that aren't just inefficient learning but almost the polar opposite of what *is* known to work. Head First Java was an attempt to challenge the typical approach to technical books by putting the learner's brain first. Head First is not at ALL an example of what learning *should* be, it is simply ONE possible implementation of a learning journey that uses what we know about the brain.
The new book is based on the science I studied *after* Head First, and the realization that it's not enough to just think about how people *learn* when they are sitting in front of a book or in a class, but the entire context in which people are having the experience of TRYING to learn and/or do something better. And much of that science is found beyond learning theory and in the psychology of motivation and also areas of the brain responsible for picking up patterns subconsciously. All of it matters.
But if there was any one single moment that changed everything, it was six years' ago when I started creating learning programs for a master horseman. He was the most EXTREME example of someone for whom even the most progressive approaches to learning design would not work. He was / is one of the best horsemen in the world, but he has been doing this since birth, really, and he has absolutely NO idea how he does what he does, and no idea how he knows what he knows. And he doesn't actually KNOW exactly what he knows. Faced with this, I realized that all of the science around learning theory I'd been using -- and using *well* (as evidenced in Head First Java) -- was failing in this case. There was no way to truly get at the Real Knowledge and his Real Skills, and no way to represent that knowledge and skills in meaningful ways.
So I had to radically change my point of view and approach and stop thinking about "teaching" and instead think about how do people acquire knowledge and skills -- at a deep level -- *without* explicit teaching? I already understood the science of expertise quite well -- I'd studied nearly six decades worth of research -- but being 'forced' into applying it without falling back into my more conventional teaching approaches was what pushed me over the edge into seeing the world differently. The book fell into place once I had finally made that shift.
I will say it's a lot like being a die-hard OO programmer and suddenly faced with FP for the first time... you can keep trying to add a few FP-ish things to your OO program, but you might not be able to TRULY see what's there until you force yourself to "think" in full FP, letting everything you know about OO fade into the background. Not forever, of course, just long enough to glimpse that world through fresh eyes, rather than as something you tape on to your existing way of approaching problems. I was struggling with FP, for example, until I forced myself to create a small game in Clojure, and making it as FP-ized as I could. Only THEN did I start to see and think a little more FPish, and then I could approach Java 8 from a different POV. I'm not advocating abandoning OO, of course, I'm just saying that you sometimes have to suspend your current way of thinking in order to "see" in an entirely new way, and THEN you can synthesize all the pieces into a new whole that is practical.
I hope that makes sense and is probably WAY more than you wanted to hear about ;)
Short version: the book itself refers to several areas of science/research on which it is based including the science of expertise and a lot of psychology.