ESET Research announces IPyIDA 2.0, a Python plug-in that integrates IPython and Jupyter Notebook into IDA
Hex-Rays IDA Pro is probably the most popular tool today for reverse engineering software. For ESET researchers, this tool is a favorite disassembler and inspired the development of the IPyIDA plug-in which embeds an IPython kernel into IDA Pro. In continuous development since 2014, we are pleased to announce the release of version 2.0. IPyIDA has a similar purpose to another plugin called IDA IPython, but with a twist: while IDA IPython only supports Windows, IPyIDA supports macOS, Linux, and Windows.
If you are already familiar with IDA Pro and IPython, skip ahead to the last section of this article on IPyIDA. If you’re not familiar with IPython, skip ahead to the middle section. Finally, if you want a quick introduction to IDA Pro, read on.
What is IDA Pro?
IDA Pro is a disassembler that translates machine code into assembly code. After uploading a file, IDA Pro disassembles it and stores the analysis in database files. IDA Pro provides various windows into the database, each of which uniquely helps the researcher find and better understand the code of interest.
Let’s take a look at some of these windows by loading the file MathLibrary.dllwhich can be created with Microsoft’s tutorial on creating a DLL file.
Exit window
The Output window displays messages about the status of a file’s analysis, error messages for user-requested operations, and the output of some plug-ins. Figure 1 shows the Output window after the first load MathLibrary.dll.
Figure 1. The IDA Output window
At the bottom of this window is an input field that accepts commands. Figure 2 shows the two default command language providers shipped with IDA since version 7.3: IDC for commands written in IDA’s native C-like language and the IDAPython plug-in for commands written in Python.
Figure 2. The input field for typing Python commands in IDA
IDA display window
The IDA View window, also known as the Disassemble window, has two viewing formats: graph view and text view. The graph view displays the program flow by dividing the functions into blocks with a single entry point and a single exit point. Figure 3 shows the graph view.
Figure 3. The IDA disassembly graph visualization
The text view provides a linear disassembly view showing virtual addresses, assembly code, and comments. Figure 4 shows the text view.
Figure 4. The IDA disassembly text display
In addition to these and many other windows provided by IDA, IDA allows you to write custom plug-ins that extend its functionality and solve practical reverse engineering problems. Let’s move on to IPython and some interesting features it offers to reverse engineers using Python scripts in IDA.
A look at IPython
While IDAPython satisfies the basic needs for running Python scripts and commands in IDA, Python enthusiasts have been caught up in IPython fever. IPython is a toolkit that offers a more interactive experience with Python. IPython uses a two process model consisting of a kernel and a client. The kernel is a process that receives commands from the client, executes them, and returns the results. The client can be any interactive console such as Jupyter Console, Jupyter Qt Console, or Jupyter Notebook.
The interactive nature of these clients comes from the set of features they add to the classic Python shell. Figure 5 shows the use of a multiline code block to define a function in IPython.
Figure 5. Using a multiline code block to define a function in IPython
Notice the syntax highlighting of integers, keywords, built-in functions, and strings.
Pressing the Form key, IPython provides a list of relevant attributes, objects or functions that can complete the code. Figure 6 shows the tab completion list functions for a string object.
Figure 6. IPython tab completion
Tab completion is richer if Jedi is installed.
IPython also provides magic functions, which are functions that are typically called with a % or %% prefix and accepts arguments with command-line style syntax. Figure 7 shows the %timeit magic function, which times the execution of a Python expression.
Figure 7. A magic function in IPython
Using the ! character at the beginning of a command line, the IPython console passes the command to the underlying system shell to execute. For example, a popular command is pip, which installs and manages packages from the Python Package Index (PyPi). Figure 8 shows the !Foo command executed by IPython.
Figure 8. Running system shell commands from IPython
IPython provides many other interactive features which can be explored in the official documentation.
IPyIDA: bring IPython into IDA
With the release of IPyIDA 2.0, writing Python scripts in IDA is easier thanks to the following advantages:
- Support for IDA on Windows, Linux and macOS
- An installation script for easy setup, even in a virtual environment
- A Jupyter Qt console running in an IDA window
- An IPython kernel that can connect to front-ends, such as the Jupyter Console, from outside IDA.
- A Jupyter kernel proxy to support opening a Jupyter notebook that reconnects to the IPython kernel with the extension %open_notebook magical function
Figure 9 shows the process of opening a Jupyter Notebook from IPyIDA.
Figure 9. The %open_notebook magic function in IPyIDA
Figure 10 shows a Jupyter console running in a terminal session outside IDA connecting to the IPython kernel in IDA.
Figure 10. A Jupyter console outside IDA reconnecting to the IPython kernel in IDA
The Jupyter Qt console choice for IPyIDA offers additional interactive features to the traditional IPython console, such as inline graphics, saving and printing the current session, and full syntax highlighting. These are explained in the official documentation for the Jupyter Qt Console.
IPyIDA also provides its own interactive features inspired by IDA. Figure 11 shows that Ctrl-clicking (Cmd-clicking on macOS) addresses or variable names in the IPython console switches the view to the virtual address in the disassemble window.
Figure 11. Ctrl-clicking a variable or address in IPyIDA jumps to the address in the IDA disassembly window
Figure 12 shows a hex dump for a byte array with non-ASCII content.
Figure 12. IPyIDA displays hex dumps
If you are new to IDA Pro, IPyIDA is a great help in getting familiar with the IDA API. If you are a veteran, IPyIDA makes Python scripting much easier, and therefore the time spent on reverse engineering hopefully more focused and fruitful.