• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Liutauras Vilda
  • Bear Bibeault
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Devaka Cooray
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Piet Souris
  • salvin francis
  • Stephan van Hulst
Bartenders:
  • Frits Walraven
  • Carey Brown
  • Jj Roberts

Tiny Python Projects: Python for other than machine learning

 
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,
I'm a Java/Gosu developer mostly working in enterprise wide web application. Recently I have seen more traction in the world of python and a lot of it revolves around machine learning. I have no background in Data science . I don't even have a CS degree from college for the matter. So I'm wondering is python a good language to learn for improving job prospects and career for someone with a Java web background and not wanting to move to data science world now.

Thanks
 
Author
Posts: 9
1
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I don't have a CS degree, either. I studied English lit/music in undergrad. 20 years later I finished my MS. In between, I learned several languages, and I credit Perl and Python as the main ones that helped me learn about this "data science" kind of thing. I mostly have used these languages to build databases and command-line programs and websites to interact and query them. I always avoided Java. I just never liked the syntax and heavy OO nature of it. I prefer to use a purely functional approach, one that relies heavily on testing and increasingly on Python's types. Python has definitely established a strong hold on data science, and I think it's worth learning just due to the number of resources that will help you learn!
 
Greenhorn
Posts: 5
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Both you guys are in trouble -- no CS degree amid a sea of prejudice.  Granted, a CS degree is a good thing.  If you have a chance, get one.  Nevertheless, some or many of us (e.g. me) have entered into the hallowed halls of machine logic by other paths, and have done 'okay' if not very well, even if we don't [or do ... or half] know and understand Liskov's Principle.  I have heard it said Einstein said he understood Relativity ... until the mathematicians got hold of it.

My first love language is Java, and now I've been somewhat forced / somewhat drawn into Python.  Rather like "Marsha Marsha Marsha" (Bing it) one hears the drumbeat "Python, Python, Python".  As a Java person, Python is frequently "strange" to me ... sometimes I feel it was invented by the Ministry of Funny Walks.  Nevertheless, many are completely gooey-eyed about Python (Pythonista's), "2 year olds" are programming in Python on Raspberry Pi's (which can be clustered using Kubernetes by the way), and in general "everything" now has a Python interface.  Python *IS* impressive, it does have some "cool" constructs, and I've learned more CS from membership on a Python learning site than I ever encountered in my lone Java programming day job.

So, Python *IS* used for "everything".  If your Java background is web use, start exploring Python by looking up "Flask" and "Django".  Python is "big" in full stack development.  Python is *NOT* just Data Sciences, AI/ML, etc.  Python is truly a "universal" solvent and "always" one of the recent top three in the TIOBE index, https://www.tiobe.com/tiobe-index/.
 
Anand Athinarayanan
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you for your responses. I have always wanted to learn a different language other than Java but never just got around to do it. Python seems different but interesting. At my work recently I had to read a bunch of files and look for specific patterns from a different file and extract the entire line matching the pattern . I'm thinking of trying it out in Python. I was able to get my request for the software approved. This is not part of my project work but just a side work that we normally use excel for. Let's see how this goes.
I'm a bit on the slower side of learning but persistent learner.
 
Ken Youens-Clark
Author
Posts: 9
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
While you could certainly write a Python program to extract lines from a file matching a pattern (I would use a regular expression, and I cover this topic in several chapters of Tiny Python Projects), you could probably just use "grep" on the command line.
 
Anand Athinarayanan
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Ken. I might have misquoted my problem. It is not a specific pattern. It's a list of unique text running into one thousand atleast . The text is a combination of alphabet and numbers example would be - XY18736272. This is the source file.
And then I have few hundred files, each file running into thousand or more lines. In some of these lines the text XY18736272 would be embedded. If I find it I need to extract the full line containing it and write to another file.
One advantage is the hundred odd files are comma separated files so I know exactly which position to look for.
At this time I have been able to just start with file reading part.
I will look at your examples also. I see there is a free preview of your book in the site. Thanks
 
Ken Youens-Clark
Author
Posts: 9
1
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ah, yes, more complicated than I understood. I feel like I can see exactly how to write it using Python (and Perl, of course). I'd be happy to work with you to show you. I think it would be <100 LOC.
 
Anand Athinarayanan
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I appreciate the offer Ken. So here is what I have so far. Won't be able to post the actual code but it's close to it. I'm using python 3.7.1



Initial tests show it is working fine but just seems very slow. I think it might be because the orderDetailsData file is on a network storage and I need to access it each time which Unfortunately I cannot copy to my machine because of "restrictions" . But what do you think and is there a better or Python way of doing it.

Have to say this, coming from a Java background I find the python libraries very useful but at the same time I couldn't think in Python yet. Keep going back to OO and SOLID principles.
 
Ken Youens-Clark
Author
Posts: 9
1
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Anand,

Here is a GitHub repo with how I would probably write this program using the patterns and principles I teach in Tiny Python Projects. I'm relying on libraries like "argparse" to create the documentation and validate the runtime arguments, the "re" module for regular expressions, and the "typing" module to ensure I'm using the types correctly. I've included how I would write and run a test suite to verify that this works. I think this program covers everything you've described. Feel free to email me and I can add you as a collaborator on the repo so we could finish it if something is missing.

https://github.com/kyclark/py-grepper

Ken
 
Ranch Hand
Posts: 66
1
Redhat Notepad Fedora Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
[I love your use of Makefile. I use Makefiles everywhere!
 
Ken Youens-Clark
Author
Posts: 9
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I think of Makefiles as runnable documentation. I would be lost without them. I'm including an appendix on Makefiles in my 2nd book, _Reproducible Bioinformatics with Python_ (O'Reilly, 2021).
 
Anand Athinarayanan
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is great Ken. Definitely much more robust and testable. I will give it a spin and get in touch if anything is not working. I'm beginning to like Python already
reply
    Bookmark Topic Watch Topic
  • New Topic