• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Does Python execution being slower than Java, affect its execution time when processing Big Data

 
Ranch Hand
Posts: 2925
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Python is slower than Java when it comes to execution time. Python is often used for Big Data processing e.g Spark etc. Does this effect performance in Big Data (with respect to the time taken )?

Thanks
 
Bartender
Posts: 217
7
Android Python Oracle Postgres Database Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The workhorse for big data applications using Python is according to my knowledge not the Python bytecode rather than binary native libraries included in Python modules. This way I assume that performance difference should not be so dramatic.
 
Saloon Keeper
Posts: 15510
363
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You should never assume that one language is faster or slower than another. Execution speed is rarely a property of the language, but rather of the platform it is running on.
 
Saloon Keeper
Posts: 7585
176
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

You should never assume that one language is faster or slower than another. Execution speed is rarely a property of the language, but rather of the platform it is running on.


Quoted for emphasis. In addition to that, to the extent than some language is actually consistently slower than some other language on a given platform, that alone still doesn't make it a knock-out criterion against it. There are likely other, more important, aspects that make this or that language preferable for a given task.
 
Monica Shiralkar
Ranch Hand
Posts: 2925
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:You should never assume that one language is faster or slower than another. Execution speed is rarely a property of the language, but rather of the platform it is running on.



Ok.I had read among the advantages and disadvantages of python that one advantage of Python is that development time is generally lesser and one disadvantage that execution is slower.
 
Tim Moores
Saloon Keeper
Posts: 7585
176
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

development time is generally lesser


That, too, is something which you simply can not say in general. I suggest to disregard whichever source these came from.
 
Monica Shiralkar
Ranch Hand
Posts: 2925
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ok.thanks
 
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you count debugging and testing time, the actual differences between languages tends to be about equal, excepting maybe assembler. WHERE you spend your time is another matter. Java, for example, requires a lot more time to be spent up front, less afterwards. Interpreted languages are generally quick to code, but more likely to blow up later in the lifecycle.

Execution speed, however is another matter. Python has addressed this in various ways, including "pyc" code and running Python in a JVM, where the same optimizations and JIT rules apply (more or less) as they do for Java.

But, as others have noted, a lot of the grunt work is not being done in Python, it's being directed By Python. If the heavy-duty work is in canned services running on a database server or GPU, then the speed of the Python part can dwindle to insignificance.
 
Monica Shiralkar
Ranch Hand
Posts: 2925
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote: Java, for example, requires a lot more time to be spent up front, less afterwards. Interpreted languages are generally quick to code, but more likely to blow up later in the lifecycle.Execution speed, however is another matter.



I was thinking this as speed of execution ( the time program takes to execute ).  If not the speed of execution , what is it ?
 
Roland Mueller
Bartender
Posts: 217
7
Android Python Oracle Postgres Database Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote: Java, for example, requires a lot more time to be spent up front, less afterwards. Interpreted languages are generally quick to code, but more likely to blow up later in the lifecycle.Execution speed, however is another matter.



My point in the answer above is that Python packages do not necessarily only Python code. Instead Python often is only a binding layer around binary code written e.g. in C/C++. This is especially true for big data handling.

Thus, using a Python module for big data means that one is NOT using pure interpreted code or only Python.

Let's examine the content of python3-numpy rpm in my Fedora box: you can see a lot of shared objects (DLLs in Windows) included.

 
Tim Holloway
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:
But, as others have noted, a lot of the grunt work is not being done in Python, it's being directed By Python. If the heavy-duty work is in canned services running on a database server or GPU, then the speed of the Python part can dwindle to insignificance.



As illustrated. For those who don't know Unix, those ".so" files in Roland's example are "Shared Object" files, which are equivalent to DLLs in Windows. Almost invariably written in C/C++ or assembler. Although I suspect Rust is also beginning to creep in.

And at the binary level, it's relatively easy to talk to the system's GPU(s), which is where the power to do things like bitcoin mining comes from. Oh, and fancy graphics, too.
 
Stephan van Hulst
Saloon Keeper
Posts: 15510
363
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I also don't think there's anything stopping anyone from writing a Python to assembly compiler.

I cringe a little when I read a Wikipedia page say something stupid like "interpreted language". Whether a language is interpreted or not is never a defining property of the language. All languages can be interpreted. All languages can be translated.
 
Tim Holloway
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:I also don't think there's anything stopping anyone from writing a Python to assembly compiler.

I cringe a little when I read a Wikipedia page say something stupid like "interpreted language". Whether a language is interpreted or not is never a defining property of the language. All languages can be interpreted. All languages can be translated.



1. Easiest way would be to write a front-end parser for the gcc compiler just like Ada did. It would compile directly to machine code - compile-to-assembler is not very common anymore.

2. Actually, languages are generally defined and designed as either interpreted (e. g. BASIC) or compiled (COBOL). Yes, you can demonstrably flip the paradigm but you generally pay a penalty. I've seen BASIC compilers and interpreted Fortran, but the reason BASIC was invented was to provide a "Fortran" that ran in an interpreted environment. As a rule, interpreted languages tend to be loosely-typed and that means that a lot of things that can be nailed down in machine code at compile time in other languages have to be deferred and/or passed on to run-time libraries. Going the other way, interpreted Fortran is likely to be tokenized just like Java bytecodes.

Java is a good case in point, in fact. Originally designed in interpreter (token) form, the addition of Just-in-Time (JIT) compiling to machine code has been a major strength for it. Since then, others have jumped on that particular bandwagon (when they don't simply jump on the JVM itself).

Then there's FORTH. FORTH is a threaded code interpreter. You define stuff or enter commands, the commands are parsed and converted to thread pointers, the interpreter then steps though the pointers, calling each thread in turn via its pointer. Note that in this context, the threads are simply bits of executable code, not asynchronous threads (usually!)

So FORTH is a lot like Python invoking machine-language library functions except that the FORTH syntax is much simpler than Python's.
 
Monica Shiralkar
Ranch Hand
Posts: 2925
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:

I cringe a little when I read a Wikipedia page say something stupid like "interpreted language". Whether a language is interpreted or not is never a defining property of the language. All languages can be interpreted. All languages can be translated.



But is it not so that interpreted languages are the ones which are compiled and run in a single step like Python and not like Java which is compiled first and then run.?
 
Tim Moores
Saloon Keeper
Posts: 7585
176
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Monica Shiralkar wrote:But is it not so that interpreted languages are the ones which are compiled and run in a single step like Python and not like Java which is compiled first and then run.?


No. Interpretation vs. compilation is a decision on how to execute programs written in a specific language. Just about any language will lend itself to different approaches on how to execute it..

Also, Java, is not"compiled and then run". It is compiled into class files, which at runtime will generally be compiled into machine code, or may sometimes be interpreted, depending on the inner workings of the JVM.
 
Tim Holloway
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Typically in the original BASIC system, you'd sit at a time-share terminal and type in the statements of your program line by line directly (not via a text editor). As you did so, each line would be converted to tokens that would become the executable program (similar to bytecodes). You could save or load your program workspace to write these tokens to disk.

If you typed the LIST command, the tokens would be de-compiled to output something similar to - but not necessarily identical to the originally typed text.

In Python, you can type commands for immediate execution, though I don't think you can save and re-load them as a program.

In Java, you generally type in the program via an external text editor to create a source file which is then compiled to bytecode. Originally, the JVM would interpret the bytecodes much like BASIC did, but then Just-in-Time (JIT) compiling was added, which allows qualifying bytecode sequences to be compiled to native executable code and stored in memory for repeated execution, since native code is almost always faster than token (byte) code.

In fact, sophisticated JVMs can take it one step further. By monitoing what's being executed, a JVM can discard a native-code sequence and re-compile the bytecodes to produce a sequence that executes faster under the current conditions. For example, conditional branch statements usually have 2 execution timings, where the branch action takes longer than the non-branch action. So moving some code and switching a branch-on-zero to a branch-on-nonzero instruction. In fact, in a multi-pipelined operation, all sorts of techniques can be used.
 
Monica Shiralkar
Ranch Hand
Posts: 2925
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Moores wrote:

Also, Java, is not"compiled and then run"



Tim Moores wrote:

compiled into class files, which at runtime will generally be compiled into machine code, or may sometimes be interpreted.



It was my misconception that in Java, 1st step is compilation into intermediate stage , i.e bytecodes, and second step is that this bytecodes can now be interpreted by JVM to executable output on any platform. And that this is what the platform independence of Java is all about (compile once run anywhere).

Now I came to know that this is wrong and I will read further on this.

Thanks.


 
Tim Moores
Saloon Keeper
Posts: 7585
176
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Monica Shiralkar wrote: It was my misconception that in Java, 1st step is compilation into intermediate stage , i.e bytecodes, and second step is that this bytecodes can now be interpreted by JVM to executable output on any platform. And that this is what the platform independence of Java is all about (compile once run anywhere).

Now I came to know that this is wrong and I will read further on this.


No, that is not wrong, it is correct. But it is not "compiled and run", which is what you said, but rather "compiled and either interpreted or compiled again and then run".
 
Tim Holloway
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Moores wrote:

Monica Shiralkar wrote: It was my misconception that in Java, 1st step is compilation into intermediate stage , i.e bytecodes, and second step is that this bytecodes can now be interpreted by JVM to executable output on any platform. And that this is what the platform independence of Java is all about (compile once run anywhere).

Now I came to know that this is wrong and I will read further on this.


No, that is not wrong, it is correct. But it is not "compiled and run", which is what you said, but rather "compiled and either interpreted or compiled again and then run".



More complex than that, even. It's compiled into bytecodes. The bytecodes are then interpreted. BUT if the JVM decides it's worth the effort, it will compile selected bytecode sequences into native code. So it's a mix of interpret and run.
 
Monica Shiralkar
Ranch Hand
Posts: 2925
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks all.
 
We can fix it! We just need some baling wire, some WD-40, a bit of duct tape and this tiny ad:
a bit of art, as a gift, that will fit in a stocking
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic