Automate the Boring Stuff with Python question

[Tomas Linhart]

Hi,

I'd like to ask a couple of questions about Python and the book itself.

1) I have been programming in different languages for about 15 years, but have no experience with Python. Can I still gain from the book, or should I pick some "Python for XXX programmers" book?
2) What's the learning curve for Python if you could compare with other languages?
3) Python seems to be the second language of choice for data scientists (R being the first). Why do you think this is true and do you think learning Python is worthy for someone who wants to start with data science?

Thanks.

Regards,
Tomas

[Al Sweigart]

Tomas Linhart wrote:
1) I have been programming in different languages for about 15 years, but have no experience with Python. Can I still gain from the book, or should I pick some "Python for XXX programmers" book?

Automate is perfect for you. The book is made for complete beginners and is split up into two parts. Part 1 covers Python basics, while Part 2 covers using various modules to automate different tasks.

So if you've never programmed before (or never programmed in Python), you can start with Part 1. Meanwhile experienced Python developers can start at Part 2.

Tomas Linhart wrote:
2) What's the learning curve for Python if you could compare with other languages?

In my opinion, Python is, without a doubt, the best language for first-time programmers because it has an incredibly gentle learning curve. The syntax is simple, the standard library API isn't that complicated, and there's hardly any boilerplate. You don't need a full-featured IDE with autocomplete to write Python (I use Sublime Text) because you can fit most of the language in your head.

Tomas Linhart wrote:
3) Python seems to be the second language of choice for data scientists (R being the first). Why do you think this is true and do you think learning Python is worthy for someone who wants to start with data science?

I don't have direct experience, but if I had to take a guess it's because data scientists (and scientists in general) are technical people who don't need to become software engineers: they view computers as tools to do practical tasks, which is something Python is excellent for. This is the reason I wrote this book using Python. Most of the time, scientists need scripts that crunch numbers or do some data processing and knowing a lot of computer science isn't really necessary. If you only have to sort a few thousand things, elegant algorithms don't matter: you can use bubble sort and it'll be "good enough" for any modern laptop.

Of course, Python has the sort() method for this, which "just works" in most cases. They aren't writing web apps that need to serve thousands of requests per second, so getting every drop of performance isn't necessary. This is an area where Python excels in. (And, if they do need performance, they can move on to learn how to write C extensions that Python can call.)

I haven't looked at R, but I can say that Python is such a good general language that it can easily fill in the functionality gaps that R doesn't cover.

[Tomas Linhart]

Thanks for answers, Al.

[Mohd Tariq]

Hi AI,

I would want to know what kind of boring stuff you meant by "automate boring stuff with python"? is it like some repetitive tasks like getting a file from FTP server and emailing it to some address etc which you have to do & generally done via cron job & shell scripting or something else? Can python automate repetitive testing like we do it from jmeter or selenium.

I am a java programmer but sometime we have to write some shell scripts or perl scripts to automate few stuff in our day to day job and as per your earlier comment if python can be very handy in this then i would definitely love to learn this as well.

Thanks
Tariq

[Mohd Tariq]

Hi Al,

I just read other threads and got an idea on what kind of automation you are talking about. I think , this would be very good for admins and even for developers as well.

Thanks
Tariq

[Al Sweigart]

Mohd: Part 2 of the book goes into automating various tasks. Here's the table of contents:

7. Pattern Matching with Regular Expressions
8. Reading and Writing Files
9. Organizing Files
10. Debugging
11. Web Scraping
12. Working with Excel Spreadsheets
13. Working with PDF and Word Documents
14. Working with CSV Files and JSON Data
15. Time, Scheduling Tasks, and Launching Programs
16. Sending Email and Text Messages
17. Manipulating Images
18. Controlling the Keyboard and Mouse with GUI Automation

[chris webster]

Al Sweigart wrote:

Tomas Linhart wrote:
3) Python seems to be the second language of choice for data scientists (R being the first). Why do you think this is true and do you think learning Python is worthy for someone who wants to start with data science?

I don't have direct experience, but if I had to take a guess it's because data scientists (and scientists in general) are technical people who don't need to become software engineers....
I haven't looked at R, but I can say that Python is such a good general language that it can easily fill in the functionality gaps that R doesn't cover.

I'm working on an R&D project with a team of developers and data scientists, and I posted some stuff about how we're using Python. Python is a great tool for us because it's got lots of stuff the data scientists need, but it's also a fully rounded general purpose programming language with excellent libraries ("batteries included") for pretty much any task. As a developer, I can put together an application e.g. to acquire data via web-scraping, then hand it over the the data scientists to adapt and build in their analytical stuff, and we can both understand the code so I can still help out with problems or work together with them on implementing different techniques e.g. machine learning. We are also starting to look at using Python with Apache Spark for large scale data analysis (there is supposed to be a Spark R API in the near future).

R has been around a long time and is specialised for statistical work. You can also use R for all kinds of tasks beyond data analysis e.g. I've seen web applications built in R and it has lots of libraries for general purpose tasks, but it seems to be especially suited to statistical work, and it's popular with many data scientists because it provides all the tools they need. R was developed by statisticians rather than computer scientists, and this is evident in the way it doesn't quite feel like most other programming languages. Some of the libraries can seem a bit eccentric e.g. there seems to be no equivalent of the "Pythonic" approach where the language itself encourages you to code in a particular style. I've learned a bit of R, as some of my colleagues are using it, but I don't find it especially comfortable yet, and I doubt I will need to use it anywhere else after this project is over.

So my own view is that R is great if it meets your needs and if your team are happy using R, but personally I think Python is much more widely applicable and is certainly easier to learn, especially if you need to share code with non-specialists. And the resources around Python are great - not just learning materials and code libraries but also the community and their welcoming approach to newbies.

But as with any programming language, you should pick the tool which seems most suitable for the job you need to do and the people who need to do it.