Git can handle binary files (such as spreadsheets, MS Word files, and image files). It would not be hard to pull up previous versions of a spreadsheet, for example, but comparing it to the current version is tricky because Git relies on third-party tools for displaying the differences. I'm not aware of any third-party tools for displaying the difference between two spreadsheets, but if one did exist, you could configure Git to use that tool. (Git for Windows sets up the third-party tool antiword
for taking the difference between MS Word files.)
As for having a non-technical team set up SSH keys to commit work, I'd say that would be hard. GitHub supports username/password authentication, and this could be used at the start.