Ask Hackaday: How Do You Python?

Python is the Arduino of software projects. It has a critical mass of libraries for anything from facial recognition and neural networks to robotics and remote sensing. And just like Arduino, I have yet to find the killer IDE for Python. Perhaps I just haven’t tried the right one yet, but it could be that I’m just doing Python wrong.

For Years I’ve Been IDLE

IDLE with interactive shell that has highlighting and code completion

I’m a Linux-only type of a guy so using IDLE for Python is a natural fit. It’s in the repositories for super quick and easy install and there’s basically zero configuration to be done. Generally speaking my preferred development environment is text editor and command line compiler. IDLE is just one step above that. You get a separate window for the shell and each Python file you’re working on. Have IDLE run your code and it saves the file, then launches it in the shell window.

For me, there are two important features of IDLE’s shell. The first is that it keeps an interactive session open after you run your Python code. This means that any globals that your script uses are still available, and that you can experiment with your code by calling functions (and classes, etc) in real time. The second desirable feature is that while using this interactive shell, IDLE supports code completion and docstring support (it gives you hints for what parameters a function accepts/requires).

But simplicity has a tough time scaling. I’m working on larger and larger projects spread over many files and the individual nature of IDLE editor windows and lack of robust navigation has me looking to move forward.

The Contenders

I’ve tried perhaps a half-dozen different Python IDEs now, spending the most time on two of them: Geany and Atom. Both are easy to install on Linux and provide the more advanced features I want for larger projects: better navigation, cross-file code completion (and warnings), variable type and scope indication.

The look of Geany brings to mind an “IDE 1.0” layout style and theme. It’s the familiar three-pane layout that places symbols to the left, code to the right, and status along the bottom. When you run your program it launches in an interactive terminal, which I like, but you lose all IDE features at this point, which I despise. There is no code completion, and no syntax highlighting.

I have been using Atom much more than Geany and have grown to like it enough to stick with it for now. I’d call Atom the “IDE 2.0” layout. It launches with a dark theme and everything is a tab.

Atom has symbol view that isn’t shown all the time. CTRL-R brings it up and it uses a search style but you can also scroll through all symbols

Atom depends heavily on packages (plugins that anyone may write). The package management is good, and the packages I’ve tried have been superb. I’m using autocomplete-python and tabs-to-spaces, but again I come up short when it comes to running Python files. I’ve tried platformio-ide-terminal, script, and runner plugins.  The first brings up a terminal as a bottom pane but doesn’t automatically run the file in that terminal. Script also uses a bottom pane but I can’t get it to run interactively. I’m currently using runner which has an okay display but is not interactive. I’ve resorted to using a “fake” python file in my projects as a workaround for commands and tests I would normally run in the interactive shell.

Tell Us How You Python

It’s entirely possible I’ve just been using Python wrong all these years and that tinkering with your code in an interactive shell is a poor choose of development processes.

What do you prefer for your Python development? Does an interactive shell matter to you? Did you start with IDLE and move to a more mature IDE. Which IDE did you end up with and what kind of compromises did you make during that change. Let us know in the comments below.

123 thoughts on “Ask Hackaday: How Do You Python?

      1. Jupyter is a Python research and programming thing. I have not seen it used with anything but Python and Python wrappers for FORTRAN and C/C++ libs. Here is a simple demo I did when it was called iPython Notebook. http://nbviewer.jupyter.org/github/ARMWorks/IPNB/blob/master/sampleIPNB.ipynb

        A very nice feature is that you can describe your algorithm or physics with LaTex and have really readable mathematics and engineering documentation. Do analysis and experiment, then take the code to the target system.

      2. There are Jupyter kernels for quite a few other languages (see https://github.com/jupyter/jupyter/wiki/Jupyter-kernels).

        As for embedded development, I don’t have a direct answer (depends largely on your workflow), but bash commands can be run in code cells by prepending them with an exclamation point, and the output is shown in the sheet, like the output from any other command. Also useful for traceability: you can export the entire sheet (with results, along with the code and markdown cells) as HTML, PDF, etc.

      1. +1 for VS Code. I use this every day for Python development and have found it has everything I need as an editor, along with great debugging support with the python plugin.

    1. vim +1
      highlighting? check
      code completion? check
      errors/warnings? check
      VCS integration? check
      all other stuff that you like from other multi-language editors/IDEs? probably also check

      I don’t really do interactive stuff, so I’m not missing an interactive console or debugger.

    1. You can also use VirtualEnv for separate environments. Works well if you’re single platform (in particular Linux). Conda definitely has the edge with the cross platform builds and such, but seems like a lot of stuff, and quite heavy, for normal use in my opinion.

  1. PyCharm community edition is, in my opinion, the best IDE for Python out there (i’m not using the professional edition).

    Built in debugger, test support, run code, profiling, pandas & jupyter notebook support, scratch pads, code refactoring tools (like variable renaming, different to CTRL+R/CTRL+H), PEP8 and other linting tools, type checking, error checking etc.

    I cannot bear using a dumb IDE these days, I’m far more productive writing with PyCharm.
    It’s also more performant than Atom.

      1. I figured the article would end with “just use PyCharm” but it didn’t. IDLE bites, Atom is a fancy editor, but PyCharm actually has integrated development features.

        1. I just downloaded it and imported one of my ongoing projects having read this thread and been reminded of its existence, and oh my gosh you are right. For my money, stick with a command line editor or go for a proper modern IDE.

    1. +1
      I have been converting coworkers to it for the last year or so now. Our git-tracked python scripts have noticeably improved in both quality and consistency. I think it is because it nudges everyone towards PEP8, highlights referencing issues like no other, and provides the best python debugging support i’ve seen for python. The smart auto-complete is a godsend when working with large and complex libraries like pyQt.

    2. Seconded for an IDE. I generally use Vim with pymode for my initial editing and creation, then switch to PyCharm with VIM key bindings for debugging and clean up. I am and always will be a command line guy but there’s just no substitute for the convenience of being able to set breakpoints, watches, etc. in a GUI and then stepping through code for debugging.

    3. I don’t do much Python, but I pretty much love PyCharm’s PHP brother, PhpStorm. i pretty much live in it and makes life a lot easier. All of their IDEs are based on the same core so the experience should be similar.

      I often work on a project with tens of thousands of file (counting the framework), and up to a dozen open source files. Performance is still good, especially if you disable code “inspections” for tools and languages you don’t use.

    4. +1
      I really like PyCharm. My main problem is that, being Java based, my McShield has to scan the archive files from time to time. Nothing like clicking on a menu or floating over underlined code and having the program freeze for three minutes while McAfee makes sure there’s no viruses. I keep the resource manager monitoring mcshield and watch for a spike when my productivity screeches to a halt. Then I can disable the on-access scanner for a few minutes (before it turns itself back on) and be on my way.

    5. Totally agree, pycharm is amazing and the community edition is free. it has debugging, and built in virtual environment to sort out dependencies and all that. For a python noob, IDLE is seriously awful!! I found PyCharm far better.

  2. I’m in Windows (the platform my employer’s CMS runs on). I hate the Idle editor’s relative inflexibility with tab/spacing so I use Sublime for text editing and Idle for testing, Not ideal but it works, I tried Atom and ended up despising it to the point I cursed it’s existence. The damn thing would crash all the time (especially with files containing very long lines like css font embedding, a requirement for my job) and I refuse to use multiple editors for different text editing tasks.

  3. The Atom IDE is functionally great you need to have an i5 and minimum of 8GB of RAM to run it. If only they developed it in C/C++ , Java or Go instead of in Javascript running in a chrome instance.

    1. Interesting, I’m running it on a Chromebook (c738t) with N3150 and 4GB RAM in Linux through Crouton. It seems to be pretty snappy for me. Maybe like 10 seconds to open the first time (packages delay launch) but after that I don’t see lag or other performance gripes.

      1. 10 second startup time is quite significant for a text editor. when i run it on my pentium laptop with 4GB of RAM not only is start up time significant, the fans start running, RAM usage is 300MB at startup (continues to grow while running) and battery life is significantly reduced.

        Text Editors should not be as resource intensive as google chrome or worse…that’s all I’m saying.

          1. IDEs like geany, vim, emacs, micro, gedit e.t.c run wonderfully on a Raspberry Pi 3. I believe that any code editor/text editor worth its salt should run smoothly on a Raspberry Pi or a Celeron CPU with 2GB of RAM.

            If you need 8GB of RAM and an i5 to run a code editor…..the problem isn’t with your hardware….but with the code editor.

  4. I python by saying “use Inline::Python “” in perl, a real language that doesn’t confuse whitespace for syntax (go on, flame away). I use Sublime Text for an IDE for most of these languages. Yes, it isn’t free. For free, gedit ain’t so bad, just doesn’t have the navigation stuff. Which of course, you need lots more for a crummy language that doesn’t make you write clearly in the first place. Everything people whine about bad programming in perl seems to be worse, to my eye, in these skiddie python programs I sometimes want to snatch a piece of (for example, on Raspberry Pi where some hardware only has support in python). I’m still waiting to see good comments in any python, ever. CPAN forever!

    1. IPython/Project Jupyter has received almost 10 million dollars (over time, not all at once) in grants, to continue its development, because It is so important and popular to the scientific community. The IPython/Jupyter notebook has transformed classrooms and learning environments, and is now the most popular programming language. MIT now uses it as the first language taught, instead of Smalltak. You can’t make these statements about Perl.

    1. I am also as much a vim guy as everybody else,
      run exclusively Linux at home,
      and love the jupyter notebooks for quick experiments up to small automation projects where Pentaho or Knime are too big,

      BUT I sure started to love VSCode last year!
      Works really well with (small) Python projects.

      One should at least have tried VSCode, for
      – its code completion
      – Git integration
      – pycharm wiggles
      – terminal access

  5. > There is no code completion, and no syntax highlighting [when running scripts in Geany].
    Geany can be configured to launch IDLE’s shell to run the script instead of opening a terminal window. It’s just a matter of going to Build > Set Build Commands and setting the execute command to “idle -r %e” (or “idle3 -r %e” for Python 3).

    1. Interestingly enough, there is mention of autocomplete in the preferences. I am sure I have seen autocomplete offer options from time to time. I also believe there is syntax highlighting.

  6. – Vim with more plugins than you can shake a stick at
    – GNU Screen (trying out tmux, but we’ll see how that goes)
    – A project management system I wrote myself, that silos projects, uses VirtualEnv to manage python deps, and even has project env-vars and loading. Spiritually based on VirtualEnv and VirtualEnvWrapper.
    – Git

    For Vim Plugins, here’s the bare minimum:
    – Vundle (https://github.com/VundleVim/Vundle.vim)
    – YouCompleteMe (https://github.com/Valloric/YouCompleteMe)
    – NerdCommenter (https://github.com/scrooloose/nerdcommenter)
    – Vim-Airline (https://github.com/vim-airline/vim-airline)

      1. Even so, don’t you two still need to have identical quantum states in every atom in your bodies to understand each other’s Perl-fu? Being like-minded is not a guarantee in deciphering a script written by a person in a write-only language.

  7. I’m a windows kind of guy… Tried idle, sublime, notepad++ and visual studio.

    Visual Studio is the best IMHO, notepad++ is also very good. IDLE would be ok, but it’s autocomplete is not strong enough to compete with VS. Watch out with VS17, python support is included in a preview version, but not the mainstream download.

  8. IDLE (and Emacs) do it backwards: they’re an editor that spawns a Python session. ipython does it right: run a temporary editor from within the Python session. You can then set your environment variable to use whatever editor you’d like. (Hint: export EDITOR=vim).

    ipython gets you syntax highlighting, autocompletion, decent help on functions, and readline. It’s totally bearable for most of the interactive stuff you need to do.

    When you want to edit a more complex function or class, ipython’s %ed magic command pulls up an editor to work on it. Hit save and quit, and you’ve got the new definition loaded. When you’ve got things right, %ed and then save the functions out to the module file that you’re working on. (Automagic saves typing the %.) For extra points, write a macro to save the current function out to the module that you’re currently developing, so that when you’re finished editing, saving the finished work is just a keystroke.

    Clearing all the cruft that you’ve got lying around from previous explorations is as easy as %run ning the module file.

    The pattern is very REPLy — work and develop the functions within Python, but using Vim (or whatever editor of your choice) for editing, and then save out the good bits once they’re tested and working. This is only bearable for editors that open up quickly, because you’re re-starting the editor with every function edit. Hence Vim and not something bloaty. Maybe if you run client/server Emacs, for instance, this wouldn’t be bad.

    Code, test interactively, edit, test, save, repeat. This workflow is very early-2000s, and all console-based, but it’s served me well for a decade. Meh. Get your IDEs off my lawn!

    I looked into Jupyter when it forked off of ipython. It’s pretty cool if that’s your cup of tea, and the multiple-users-per-session feature looks killer for pair programming. The notebooks idea is great for instructional stuff or demos.

    1. I have been working on a python project and iPython was my interactive of choice with Atom being editor.
      After reading lots of good comments here about PyCharm, I downloaded and run it. First impression is the IDE look more complicated than it ought to be, not to say that it seems slow but maybe it’s because my laptop is slow.
      Nah, I’ll come back to Atom and iPython. Thanks for the tip about ipython %ed editor, never knew that it had. I just thought iPython was a more advanced interactive python interpreter.

  9. Emacs, hands down. Esecially with Org Mode, which allows you to mix documentation and code (literate programming), for when you have to explain what you are doing (to yourself, to others).

  10. I use PyCharm professional edition. I actually purchase the entire suite of Jetbrains tools every year, so I have the full suite of IDEs available to me. I also use almost all of them, so it was a good investment for me.

  11. Totally PyCharm.. A great IDE for python. Love it.

    If forced to say something bad about PyChrm, it would be that it is fairly slow on slow computers, being written in Java.

  12. VIM all the way, there was a time I used emacs with evil mode, but I switched back to vim for the sake of versatility, and the fact that is usable almost everywhere that there is unix or linux system.

  13. For me, my “IDE” is FVWM. Yes that’s a window manager, but it’s how I interact with the text editor windows (vim-qt), terminal sessions (lilyterm) and web browser (Firefox).

    I use this IDE for full-stack development: anything from KICAD PCB design; device firmware in C (including U-boot and the Linux kernel); userspace software coded in C, C++, Python, Perl, etc; database schema development with SQLite3, PostgreSQL; web services in Python, NodeJS, PHP, C, C++; and front-end design with command line, ncurses, Qt and HTML5/JavaScript.

    As an IDE, I find it very general. The desktop page is divided into four quadrants, and I have key bindings that can resize a window to occupy any of those four quadrants, including spanning two or all four. I find if I keep the text lines in the window to about 72-75 characters, I can fit two files side-by-side with no text wrapping. (My laptop at work runs a screen resolution of 1366×768. My home one is 1280×800, which makes this a tad squashed, but not unworkable.)

    I could emulate the classic 3-pane layout of an IDE if I want to, but usually I just have editor windows open. Often two side-by-side: I prefer this to tabs as I can refer to one file whilst editing another. Or the other side can be a PDF datasheet, a browser page, terminal window… anything I need at the time.

    For python within the terminal session, I use ipython. If I want to have the “IDLE” feature of seeing what the global variables are at the end, I can run ipython -i scriptname.py and it pretty much drops me into an interactive Python shell at the end.

    For what it’s worth, standard python -i scriptname.py will do this too, but I find ipython nicer to use.

  14. VScode is the shiznit. I’ve used Spyder, jupyter, sublime, even nano. Didn’t think I would fall for a Microsoft product but it’s so good. Easy to customize with a json file, rich extension system, reads all kinds of file typs, git integration, pretty like atom but ten times faster like sublime text. Can even run jupyter inside of it.

  15. I avoid Python where possible, but my editor of choice for all coding in various languages, is kate.

    Hasn’t always been, I have gone though jedit, nedit, gedit, quanta, emacs, xemacs, eclipse and so forth, but kate has stuck around perhaps for the longest time.

    It’s simple, it gets the job done, it has the right plugins, it doesn’t force you into some annoying project based paradigm if you want to just work from a file-tree you can work from a file-tree.

    As I say I don’t use Python enough to know how much bell-and-whistle kate adds for it (code completion etc), but I imagine what you need is there.

    Aside: I really hate how “Atom” thinks that it knows better than me and my operating system preferences how it should display it’s gui. Black with totally non-system widgets, ugh. Use a proper toolkit.

  16. You guys are ignoring the embedded interactive sector. We had (still have for a month or so) Pymakr – https://github.com/pycom/Pymakr
    It was just making the transition to working on all embedded boards but now its been EOL’d. :(
    Micropython uses Mu (too dedicated), Micropython and Circuitpython(adafruit) use multiple components.

    Some talk that Atom is the way to go…

    Does anyone have any experience with Pycharm on an embedded controller – I.e. has to do the download and open a remote repl shell – sometimes via webrepl(wifi) rather than serial…?

    FWIW I’m still using Notepad++.. sigh…

  17. I use VScode. I used atom for a while but he startup times got worse so I binned it. One thing I hate about most editors today is the fact that you can’t print. Sometimes it’s nice to print some code out to read away from the PC but the editors don’t want you doing that.

  18. PTVS is awesome. Ie Python Tools for Visual Studio. Use free community edition of VS. SUPPORTS FULL DEBUGGING , step , play, etc right in the GUI. Supports vurtualenv as well., runs with 2.7 and or 3.x versions just fine. Auto complete (Aka intelli-sense ) rocks too.

  19. I just came across SPE ( Stani’s Python Editor ) in my Linux distribution. It fit my immediate needs and seems quite good. One of its more interesting features is that when run from within Blender it allows you to browse Blender’s 3D constructs… or so it says.. as I haven’t tried.
    I’ll look into some of the other options mentioned here too but would be interested in hearing what others think of SPE.

  20. I’m​ a grown up, so I don’t python unless I have to :-p
    For Windows users I would recommend Visual Studio free edition and for Mac/*nix users Visual Studio Code (also free)

  21. Great thread – with lots of options to try.
    Currently I use IDLEX (IDLE with some extra stuff), Geany, UltraEdit, Visual Studio. I generally like JetBrains stuff; work gives me PHPStorm. Didn’t realize there was a ‘community’ edition of PyCharm.

  22. I use Jupyter to start most projects. If I need debuggering. I’ll switch over to PyCharm. Jupyter with ipwidgets offers a lot of customization. If github integration was better, I might just stick with Jupyter. It keeps improving so quickly.

  23. “Python is the Arduino of software projects”

    No. Python is industrial grade. Python works its butt off on all major platforms. But if you are referring to ‘micro’ or ‘circuit’ Python, then yes, it is Ardy-grade stuff that is probably ill-suited for other than hobby work. And fwiw, SCons and C and Python are The Force that surrounds our hardware designs, and binds us the logic of circuits. The java and ardy stuff is of the dark side – it represents the emotive and irrational path.

  24. Having not used python much, what’s the advantage to any of the non-general purpose editors, over that of a multi-tabbed terminal and one of the general purpose editors which have syntax highlighting. I usually use vim, but I’m also thinking of emacs, kate, and any other gp text editor.

    For someone who doesn’t program purely in python, try to convince me that some of these have useful features that I’ve been missing out on. The only one I’ve seen so far is the one which allows you to browse blender objects.

    1. The advantage is whatever it does to increase your productivity and/or to standardize your ‘interface’ to your development tool chain.

      vi/Vim or Emacs are generically wonderful if they get things done.

      Complex code completion, doc string utilization, format control, and debuggers are all important for larger projects, so the editor’s ability to talk to this stuff becomes critical for the more complex task sets. Per your description, this is not in your problem space, so Vim away without any plugins, or any remorse. Or use ed…

  25. Pycharm – it’s awesome. Supports a lot of different version control systems ,auto suggests well, and even recommends best coding practices. Plus it runs on a lot of different platforms and is aesthetically pleasing. There is a paid and a free community version, the community version is just fine for me presently. https://www.jetbrains.com/pycharm/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s