Merge pull request #1166 from qilingframework/dev

Getting ready for 1.4.3

.github/workflows/build-ci.yml

@@ -10,7 +10,7 @@ jobs:
       fail-fast: false
       matrix:
         #os: [windows-2019, macos-10.15, ubuntu-18.04, ubuntu-20.04]
-        os: [windows-2019, ubuntu-18.04, ubuntu-20.04]
+        os: [windows-latest, ubuntu-18.04, ubuntu-20.04]
         python-version: [3.8, 3.9]
         exclude:
           - os: ubuntu-18.04

.github/workflows/giteesync.yml

@@ -5,6 +5,7 @@
  jobs:
    deploy:
      runs-on: ubuntu-latest
+     if: github.repository_owner == 'qilingframework'
      steps:
      - uses: actions/checkout@v2
        with:

CREDITS.md

@@ -3,27 +3,27 @@
 
 #### Founder
 
-- LAU kaijern (xwings) <[email protected]>
+- LAU kaijern (xwings) <kj_at_qiling_io>
 
 
 #### Advisor
 
-- NGUYEN Anh Quynh <[email protected]>
+- NGUYEN Anh Quynh <aquynh_at_gmail_com>
 
 
 #### Core Developers Crew
 
-- Earl MARCUS (klks84) [email protected]
-- WU chenxu (kabeor) <[email protected]>
-- KONG ziqiao (lazymio) <[email protected]>
-- YU zheng (dataisland) <[email protected]>
-- Eli Cohen Nehemia (elicn) <[email protected]>
+- Earl MARCUS (klks84) <klks84_at_gmail_com>
+- WU chenxu (kabeor) <kabeor_at_qiling_io>
+- KONG ziqiao (lazymio) <mio_at_lazym_io>
+- YU zheng (dataisland) <dataisland_at_outlook_com>
+- Eli Cohen Nehemia (elicn) <elichn_at_gmail_com>
 
 
 #### CI, Website，Documentations, Logo & Swags
 
-- FOO Kevin (chfl4gs) <[email protected]>
-- SU muchen (Mirai Suu) <[email protected]>
+- FOO Kevin (chfl4gs) <chfl4gs_at_qiling_io>
+- SU muchen (miraisuu) <suu_at_iling_io>
 
 
 #### Key Contributors (in no particular order)
@@ -52,14 +52,17 @@
 - bambu
 - madprogrammer
 - danielmoos
+- sigeryang
+- bet4it
+- nullableVoidPtr
 
 
 #### Legacy Core Developers
 
-- DING tianze (D1iv3) <[email protected]>
-- SUN bowen (w1tcher) <[email protected]>
-- CHEN huitao (null) <[email protected]>
-- YU tong (sp1ke) <[email protected]>
+- DING tianze (D1iv3)
+- SUN bowen (w1tcher)
+- CHEN huitao (null)
+- YU tong (sp1ke)
 
 
 #### Demigod team (https://groundx.io/demigod)

ChangeLog

@@ -1,5 +1,58 @@
 This file details the changelog of Qiling Framework.
 
+------------------------------------
+[Version 1.4.4]: July XX, 2022
+
+
+------------------------------------
+[Version 1.4.3]: June 1st, 2022
+
+New features:
+- Introduce PowerPC architecture support (#1140)
+
+Improvements:
+- Fix fuzzing for tendaac15 (#1096)
+- Update unicorn version to 2.0-rc6 (#1100)
+- Implemented a few more Windows msvcrt functions (#1102)
+- Minor PE Loader fix (#1104)
+- Minor quality changes (#1106)
+- Fix cacheflush syscall typo (#1115)
+- Improvements and fixes for Windows and PE (#1118)
+- Add vm_context to EVM hooks (#1119)
+- Load interpreter segments with correct perms and vaddr (#1120)
+- Fix mistakes in fuzz_x8664_linux binary (#1121)
+- Add EVM ABI helpers, fix EVM DBG stack view (#1123)
+- Fix regression caused by missing exception handling when opening socket (#1124)
+- CI improvement (#1128 #1134)
+- Add macho load command 'LC_LOAD_WEAK_DYLIB' support (#1133)
+- Fix breakage of non-Windows binary emulation on Windows host (#1143)
+- Remove misused region bound check of unmap_all (#1144)
+- Change deprecated interfaces of IDA (#1145)
+- Use importlib to retrieve package version (#1146)
+- New and improved gdbserver (#1148)
+- Rewrite package data reading (#1150)
+- Misc improvements (#1154)
+- Fix memory exhaustion problem caused by the logger (#1161)
+
+Contributors:
+- wtdcode
+- aquynh
+- elicn
+- xwings
+- cq674350529
+- elicn
+- TheZ3ro
+- bet4it
+- chinggg
+- kabeor
+- chfl4gs
+- profiles
+- OlfillasOdikno
+- nmantan
+- machinewu
+- nullableVoidPtr
+- Phat3
+
 
 ------------------------------------
 [Version 1.4.2]: Feb 13th, 2022

Dockerfile

@@ -23,7 +23,7 @@ WORKDIR /qiling
 RUN apt-get update \
   && apt-get install -y --no-install-recommends unzip apt-utils \
   && rm -rf /var/lib/apt/lists/* \
-  && pip3 install wheels/*.whl \
+  && pip3 install --no-deps wheels/*.whl \
   && rm -rf wheels
 
 ENV HOME /qiling

README.md

@@ -8,10 +8,12 @@
 <img width="150" height="150" src="https://raw.githubusercontent.com/qilingframework/qiling/master/docs/qiling2_logo_small.png">
 </p>
 
+[Qiling's usecase, blog and related work](https://github.com/qilingframework/qiling/issues/134)
+
 Qiling is an advanced binary emulation framework, with the following features:
 
-- Emulate multi-platforms: Windows, MacOS, Linux, BSD, UEFI, DOS, MBR, Ethereum Virtual Machine
-- Emulate multi-architectures: X86, X86_64, Arm, Arm64, MIPS, 8086
+- Emulate multi-platforms: Windows, MacOS, Linux, Android, BSD, UEFI, DOS, MBR, Ethereum Virtual Machine
+- Emulate multi-architectures: 8086, X86, X86_64, ARM, ARM64, MIPS, RISCV, PowerPC
 - Support multiple file formats: PE, MachO, ELF, COM, MBR
 - Support Windows Driver (.sys), Linux Kernel Module (.ko) & MacOS Kernel (.kext) via [Demigod](https://groundx.io/demigod/)
 - Emulates & sandbox code in an isolated environment
@@ -88,55 +90,55 @@ Please see [setup guide](https://docs.qiling.io/en/latest/install/) file for how
 
 #### Examples
 
-- Below example shows how to use Qiling framework to emulate a Windows EXE on a Linux machine
+- The example below shows how to use Qiling framework in the most striaghtforward way to emulate a Windows executable.
 
 ```python
-from qiling import *
-
-# sandbox to emulate the EXE
-def my_sandbox(path, rootfs):
-    # setup Qiling engine
-    ql = Qiling(path, rootfs)
-    # now emulate the EXE
-    ql.run()
+from qiling import Qiling
 
 if __name__ == "__main__":
-    # execute Windows EXE under our rootfs
-    my_sandbox(["examples/rootfs/x86_windows/bin/x86_hello.exe"], "examples/rootfs/x86_windows")
+    # initialize Qiling instance, specifying the executable to emulate and the emulated system root.
+    # note that the current working directory is assumed to be Qiling home
+    ql = Qiling([r'examples/rootfs/x86_windows/bin/x86_hello.exe'], r'examples/rootfs/x86_windows')
+
+    # start emulation
+    ql.run()
 ```
 
-- Below example shows how to use Qiling framework to dynamically patch a Windows crackme, make it always display "Congratulation" dialog
+- The following example shows how a Windows crackme may be patched dynamically to make it always display the "Congratulation" dialog.
 
 ```python
-from qiling import *
+from qiling import Qiling
+
+def force_call_dialog_func(ql: Qiling):
+    # get DialogFunc address from current stack frame
+    lpDialogFunc = ql.stack_read(-8)
 
-def force_call_dialog_func(ql):
-    # get DialogFunc address
-    lpDialogFunc = ql.unpack32(ql.mem.read(ql.reg.esp - 0x8, 4))
     # setup stack memory for DialogFunc
     ql.stack_push(0)
-    ql.stack_push(1001)
-    ql.stack_push(273)
+    ql.stack_push(1001)     # IDS_APPNAME
+    ql.stack_push(0x111)    # WM_COMMAND
     ql.stack_push(0)
+
+    # push return address
     ql.stack_push(0x0401018)
-    # force EIP to DialogFunc
-    ql.reg.eip = lpDialogFunc
+
+    # resume emulation from DialogFunc address
+    ql.arch.regs.eip = lpDialogFunc
 
 
-def my_sandbox(path, rootfs):
-    ql = Qiling(path, rootfs)
+if __name__ == "__main__":
+    # initialize Qiling instance
+    ql = Qiling([r'rootfs/x86_windows/bin/Easy_CrackMe.exe'], r'rootfs/x86_windows')
+
     # NOP out some code
     ql.patch(0x004010B5, b'\x90\x90')
     ql.patch(0x004010CD, b'\x90\x90')
     ql.patch(0x0040110B, b'\x90\x90')
     ql.patch(0x00401112, b'\x90\x90')
+
     # hook at an address with a callback
     ql.hook_address(force_call_dialog_func, 0x00401016)
     ql.run()
-
-
-if __name__ == "__main__":
-    my_sandbox(["rootfs/x86_windows/bin/Easy_CrackMe.exe"], "rootfs/x86_windows")
 ```
 
 The below Youtube video shows how the above example works.
@@ -221,6 +223,7 @@ Contact us at email [email protected], or via Twitter [@qiling_io](https://twitter.
 
 Please refer to [CREDITS.md](https://github.com/qilingframework/qiling/blob/dev/CREDITS.md)
 
+
 ---
 
 #### This is an awesome project! Can I donate?

examples/crackme_x86_linux.py

@@ -16,14 +16,11 @@
 
 class Solver:
     def __init__(self, invalid: bytes):
-        mock_stdin = pipe.SimpleInStream(sys.stdin.fileno())
-        mock_stdout = pipe.NullOutStream(sys.stdout.fileno())
-
         # create a silent qiling instance
-        self.ql = Qiling([rf"{ROOTFS}/bin/crackme_linux"], ROOTFS,
-            verbose=QL_VERBOSE.OFF, # thwart qiling logger output
-            stdin=mock_stdin,       # take over the input to the program using a fake stdin
-            stdout=mock_stdout)     # disregard program output
+        self.ql = Qiling([rf"{ROOTFS}/bin/crackme_linux"], ROOTFS, verbose=QL_VERBOSE.OFF)
+
+        self.ql.os.stdin = pipe.SimpleInStream(sys.stdin.fileno())  # take over the input to the program using a fake stdin
+        self.ql.os.stdout = pipe.NullOutStream(sys.stdout.fileno()) # disregard program output
 
         # execute program until it reaches the 'main' function
         self.ql.run(end=0x0804851b)
@@ -32,7 +29,7 @@ def __init__(self, invalid: bytes):
         #
         # since the emulation halted upon entering 'main', its return address is there on
         # the stack. we use it to limit the emulation till function returns
-        self.replay_starts = self.ql.reg.arch_pc
+        self.replay_starts = self.ql.arch.regs.arch_pc
         self.replay_ends = self.ql.stack_read(0)
 
         # instead of restarting the whole program every time a new flag character is guessed,
@@ -92,21 +89,26 @@ def replay(self, input: bytes) -> bool:
 
         return False
 
+def progress(msg: str) -> None:
+    print(msg, end='\r', file=sys.stderr, flush=True)
+
 def main():
-    idx_list = (1, 4, 2, 0, 3)
-    flag = [0] * len(idx_list)
+    flag = bytearray(b'*****')
+    indices = (1, 4, 2, 0, 3)
 
-    solver = Solver(bytes(flag))
+    # all possible flag characters (may be reduced to uppercase and digits to save time)
+    charset = string.printable
 
-    for idx in idx_list:
+    progress('Initializing...')
+    solver = Solver(flag)
 
-        # bruteforce all possible flag characters
-        for ch in string.printable:
-            flag[idx] = ord(ch)
+    for i in indices:
+        for ch in charset:
+            flag[i] = ord(ch)
 
-            print(f'Guessing... [{"".join(chr(ch) if ch else "_" for ch in flag)}]', end='\r', file=sys.stderr, flush=True)
+            progress(f'Guessing... {flag.decode()}')
 
-            if solver.replay(bytes(flag)):
+            if solver.replay(flag):
                 break
 
         else:
@@ -116,3 +118,5 @@ def main():
 
 if __name__ == "__main__":
     main()
+
+# expected flag: L1NUX