The GIL means that only one thread can be in a state of execution at any point in time. That isn't really multi-threaded.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Python CPU-bound multiprocessing benchmarks: 3 Intel Macs (+ Parallels on one) vs. the 16” M1 Pro MBP
- Thread starter Menneisyys2
- Start date
- Sort by reaction score
C extensions don’t get blocked by the GIL (numpy, for example). Some muti-threaded workloads are used for IO, which has threads mostly waiting anyway. It’s multi-threaded enough to be incredibly useful, even if it isn’t “true” multithreading. Again, I’m valuing my productivity over the semantics.
As Xiao_Xi said: "Every data scientist faces the eternal two-language programming problem."
Python is commonly used on Wall Street but the core analytic code is normally written in C++.
By the people writing those libraries, sure, but the rest of us mere mortals just import their libs and code in python.
UPDATE (09/Jan/2022 04:22CET): I’ve just posted a separate thread on hacking PyCharm to use the ARM JDK instead of the built-in x86 one for a MAJOR speedup - now it seems to be totally usable!!!
forums.macrumors.com
MAJOR speedup (Win11 ARM, Parallels): Hacking PyCharm to use the ARM JDK instead of the built-in x86 one
Note: at work, I need to do some serious work in PyCharm under Windows on my Mac. This is why I’m more than interested in making PyCharm work in the ARM version of Windows 11. In my previous thread (...
forums.macrumors.com
I mean, most Python libraries that include fast C code (e.g. Numpy) still contain a heck of a lot of pure Python code wrapped around it, so pure Python performance is still important for those of us who use those libraries.
As an aside, Numpy got native support for the M1’s AMX coprocessor mid-2021 via Accelerate.framework and recently got some big ARM optimizations merged for common functions, so Apple Silicon is sitting pretty right now for those of us who have to work with big matrices. ?
This is a technicality since most people use the standard interpreter but not all Python interpreters have a GIL (or have one that can be disabled).
And even if Python itself is not primarily built for speed (not without calling compiled code) but for ease of programming, as others have said: increasing the speed of the Python code is still nice when that's what you are using!
For Stefan Karpinski, a co-creator of the Julia programming, the social barrier, a wall between users and developers, is one of the main drawbacks of the two-language programming problem.
Source: https://github.com/scivision/python-performance
Speed of Matlab vs Python vs Julia vs IDL
Speed of Matlab, Python using Numpy, Numba, and PyCUDA, Julia, and IDL are informally compared.
www.scivision.dev
If you want to improve the performance of your Python code, you can use Pypy to improve pure Python code or Numba to improve C-libraries based Python code.
In order to learn Python, I ported some of my old Perl scripts to it. My first impressions were mostly positive since Python's a far better language for both writing and reading code -- its "programmer ergonomics" are vastly superior. (There's things I don't like, mostly the incredibly dumb syntactically significant indentation, but when you're comparing to Perl, all the other usability factors easily win.)
But when I benchmarked the subset of my scripts designed to process tons of data on some of my real datasets, the old Perl version always won. Sometimes by a lot. Keep in mind, even at that time (several years ago), Perl was an almost-dead language whose interpreter hadn't seen substantial updates for years.
If you're not doing the kind of numeric computations libraries like numpy and friends help out with (by doing the real work outside Python), there's no way around it. Python's just horribly slow. You can't expect any bytecode interpreted scripting language to be particularly fast, of course, but it's still quite bad relative to its peers.
When I first started Perl was the language of bioinformatics and Python was just on the upswing.
I am so very very glad Python replaced it. But I have to admit the indentation system is really bad - especially when dealing with tabs vs spaces. That gets really annoying.
If you’re needing to run Python scripts within Power Bi for example, then Python performance in a windows VM would be important if you’re running a Mac.
The lack of curly braces makes Python code cleaner.
VS Code makes very easy to convert indentation to spaces or tabs.
Python is still slow when in comparison to compiler languages. There’s no way around it.
The difference now vs 10 years ago, is that computers now mean that Python being “slow” is still fast enough.
This is a “to each his own” style but I disagree. I find it less readable and more annoying to write.
Yes there are systems to help. It’s still annoying to have to deal with it, especially when prototyping quickly and combining multiple sources together. Spaces vs tabs and the amount of each shouldn’t be a runtime error.
It’s worth noting that Python is alone in this regard amongst the “big” languages and none of the newer “big” languages developed since Python have adopted indentation as the source of delineating code blocks. Given that, I think most see it as a design flaw.
Don’t get me wrong, I like Python in most respects. Just not that one.
Last edited:
Since it's become so popular among scientists, I'm glad Python forces you to write properly indented code. It may or may not be more readable than properly indented and structured C/C++ code (for example), but it sure is more readable than 'freestyled' C/C++ code.This is a “to each his own” style but I disagree. I find it less readable and more annoying to write.
Yes there are systems to help. It’s still annoying to have to deal with it, especially when prototyping quickly and combining multiple sources together. Spaces vs tabs and the amount of each shouldn’t be a runtime error.
It’s worth noting that Python is alone in this regard amongst the “big” languages and none of the newer “big” languages developed since Python have adopted indentation as the source of delineating code blocks. Given that, I think most see it as a design flaw.
Don’t get me wrong, I like Python in most respects. Just not that one.
Maybe so, but speaking as a scientist: not worth it
I get why it was designed this way, but I also get why no one else followed suit
Can you explain it?
If you indent your code, the only difference between Python and other programming languages that don't enforce indentation is the lack of curly braces. Why do you prefer C-style code instead of Python-style code?
1) Prototyping: This is the biggest one. As I alluded to earlier, if I’m prototyping code, testing it quickly, combining multiple code sources together, I just want to go quickly and I don’t care about formatting. Python forces me to care whether I want to or not. Again I shouldn’t get runtime errors from poor formatting of prototype code which is more or less designed to be thrown away. The same is doubly true when actually typing in the interpreter in interactive mode (especially on a headless system).
2) Readability: proper formatting and readability is subjective and I don’t always want Python to force a particular indentation format on my code. But more importantly, for nested (and long) code blocks relying on white space to guide where the code block ends is just poor UX. In contrast a symbol that states a code block is ending gives you an immediate visual representation of how many code blocks are ending at this location and thus where you are in your own code.
The above are my main gripes. Others may have more that I can’t think of off the top of my head and maybe I’ll add more. But as I said earlier: I understand why no other major language does this including ones developed after Python.
Copy code can become a nightmare very easily. Every time I copy code from the Internet, especially Google's code, I need to fix the indentation.
Someone got so pissed off of Python indentation that he wrote a transpiler to write Python code with curly braces.
Why Python Indentation sucks and what can be done about it!
Every programmer who's had experience with C/C++/Java (and many more) understands the pain one has to go through with forced Python Indentations! Now before you scroll down believing this to be one of those juveniles crying over nothing significant, let me tell you that this isn't one of those typic
I'm in charge of a research group at my uni, and we have undergrads coming in and out constantly with various levels of programming competence. Without question, projects that are Python-based are easier to deal with in this context, even when considering that the vast majority of students have zero Python experience but at least some experience with C or x86 assembly by this point. It may be that the C/++ project(s) we have are just too complex at their level, but I'm convinced that it's simply faster/easier to prototype with higher-level languages. You can almost just keyboard-mash and get what you're looking for in Python...
No. I’ve personally looked into Julia, but what you often see with research projects is that you’ll have very few students intimately familiar with the code base, and perhaps a few familiar with technical details, but not enough spread to rework an entire project, especially without any commercial backing to do so.
As an example, one project that is primarily c++ consists of roughly 1M lines of code and contains work from three phd students that have since graduated and moved on. There’s almost no usage of the system outside of research atm, and the targets are niche.