GSoC 2022: Qiling's Debut!

[wtdcode]

Hello!

It is the first time we participate in Google Summer of Code and we are offering interesting ideas and projects for students of all levels.

As our README suggests, Qiling is an advanced binary emulation framework with lots of exciting features. Although it may look complicated and hard, we also prepare some ideas for starters to join our community. Just clone our repository and pick one idea below, you are ready to roll!

If you feel interested, please join our GSoC group to have a brainstorm! Also feel free to leave your questions in each idea below.

[wtdcode]

Bridge with other static analysis software

• Estimated Difficulty: Medium
• Expected skills: Python, Binary Analysis, Assembly

Qiling does a good job in dynamic analysis while lacking the ability to do static analysis for a long time. We hope to provide our users with lots of high-level concepts like stack frame, CFG and even pseudo code. Considering the principle of not inventing wheels, we are exploring the possibility to integrate Qiling with several known static disassembly sofwares:

• Radare2: Via python bindings r2libr.
• IDA Pro: Note the integration is the other way around instead of making Qiling an IDA plugin. We would like to run QIling outside IDAPython environment, which requires an IDA bridge. A possible helpful issue: IDA Integration: Work with IDA 7.0 (specifically 7.5+) pwndbg/pwndbg#844
• Ghidra: Possible via ghidra_bridge.
• RetDec: Maybe retdec-python

Depending on your ability, you may accomplish one or more integration. You may start from:

pip install r2libr qiling

to have a try!

[wtdcode]

Clarification: This idea is to add new interface like ql.static.cfg or ql.extensions.r2.esil or ql.decompiler.decompile, not to write another plugin for r2, ida or ghidra. We would like to use these analysis softwares as external libraries.

[Stolas]

Sounds like BinaryNinja is perfect for this task.

[wtdcode]

Sounds like BinaryNinja is perfect for this task.

Just state it clearly (and perhaps have some initial attempt) in the proposal if anyone would like to have a try for some other softwares.

[wtdcode]

Improve robustness and usability of the project

• Estimated Difficulty: Easy or Medium depending on you.
• Expected skills: Continuous Integration, System Administration, Github Action

The development of Qiling is always on a fast ring, however, this sometimes causes confusion among our users due to uncovered tests, bad documents or bad code hint etc. Thus, we are expecting some help to make everything tidy and intuitive and improve the robustness of the whole project. Roughly speaking, it includes:

• CI Improvements: Qiling provides lots of examples, however not all of them are well tested in CI and this often causes regression between releases.
• Documents: Unfortunately, current Qiling documents are arranged badly. Maybe something like pydoc can help? And we need more tuition, not simple API description. Also, the documents for developers and contributors are lost.
• Unit tests: Many Qiling features should have unit tests to avoid regression.
• Features test: Ensure all Qiling instrument features works on all possible architecture and systems combination.

Basically, anything which improves the robustness can be counted. You may start by reading our CI and tests.

[wtdcode]

Documents:

• Usage about different attributes like ql.regs (ql.arch.regs in a sooner version)
• Usage about system concepts, like files, stdin, stdout, API hijack etc.
• To be added...

[wtdcode]

Gap between Uc2 and QIling

• uc_ctl
• context r/w regs...

[kk-kd]

Can you clarify what's expected for "CI improvements"? I noticed that there's already a CI built for examples.

[wtdcode]

Can you clarify what's expected for "CI improvements"? I noticed that there's already a CI built for examples.

Not all of our samples are tested against CI, like dlink fuzz case, and we are constantly facing regression bugs.

[wtdcode]

Pong!

• Estimated Difficulty: Medium or Hard depending on you.
• Expected skills: Python, Operating System, Computer Architecture, Reverse Engineering, System Programming

Qiling supports intel 8086 architecture and partial DOS emulation, which is not enough to run a pong game. It would be cool to do it! The difficulty of this idea varies depending on the concrete work, which MAY includes:

• Old days interrupt implementation
• Some hardware implementations like timeclock
• PE file parsing and some dos specific stuff

If you want to challenge yourself, we are more than happy if you can run Doom!

[wtdcode]

Networking Improvements

• Estimated Difficulty: Medium or Hard depending on you.
• Expected skills: System Programming, Network(Socket) Programming

As is often the case, malware may have many network activities. However, current Qiling has really not very well support for networking, especially non-blocking network IO. This may include and need:

• Kernel source reading (as you are implementing syscalls!)
• Multithread improvements.
• Maybe proxy and network stream API support.
• Extra patience (remember you are doing very very low-level work!)

Note not all of the points above need to be done in one project. If you are interested, we may discuss it together to figure out what to focus on.

[wtdcode]

macOS/Windows/Android Improvements

• Estimated Difficulty: Medium
• Expected skills: System Programming, System Administration

Currently, among all operating systems Qiling supports, Linux should be the most complete one while the others need a bit more polish. This idea mainly involves more syscalls implementation, tests, examples etc. Possible tasks:

• Implement more syscalls to run more binaries (especially for windows).
• Make Qiling able to execute native binaries on native macOS rootfs /.
• Better support for analyzing Android JNI code libraries.

The difficulty of this task largely varies depending on what you prefer to do. Contact us if you would like to choose this one.

[wtdcode]

Run xv6

• Estimated Difficulty: Hard
• Expected skills: Almost everything in computer systems.

As you may already know, Qiling is based on Unicorn Engine and Unicorn forks from QEMU. In fact, QEMU has both system emulation and usermode emulation and Unicorn only forks the former one. However, so far what Qiling does is more like usermode emulation and thus the idea is simple: "Can Qiling do system emulation by running a kernel?". We expect the talented you to prove this!

Considering Linux kernel is too complex to emulate, we hope you can use Qiling to emulate xv6 kernel and its shell.

This idea needs a thorough understanding of the operating system, computer architecture, and the whole Qiling (and even Unicorn!) codebase. Roughly speaking, you need to do:

• Learn how Unicorn/QEMU works to emulate instructions, syscalls and so.
• Understand how operating system is booted, how memory is mapped...
• Know the boundary between emulated kernel and your code (handle interrupts, emulate hardware...), fix the gap between current codebase design and the better one (Qiling codebase is largely designed for usermode emulation!).