Release v1.0.4

CHANGELOG.md

@@ -0,0 +1,75 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.4] - 2024-04-06
+
+### Added
+
+- Added the `--force` option to use the default vendor, bypassing analysis.
+- Progress information for analysis stages.
+- Support for Android 14.
+- Error message for using SDK version 34 and above without an XML functions file.
+- Documentation links for certain error messages.
+
+### Changed
+
+- Switched from Frida to ADB for listing processes due to a [Frida issue](https://github.com/frida/frida/issues/2669).
+- Optimized process search to improve performance.
+- Improved error reporting when the Widevine process is not detected.
+
+### Fixed
+
+- Fixed compatibility with buggy `frida-server` versions by using direct PID attachment.
+- Updated the script handling for non-standard version scenarios.
+
+## [1.0.3] - 2024-04-01
+
+### Added
+
+- Environment check for ADB and automatic start if not running.
+- Extraction function support for SDK version 34 and above.
+- Simplified command-line argument processing.
+
+### Changed
+
+- Enhanced error handling to avoid Frida library hook errors.
+- Transitioned from using symbols to functions for better clarity and efficiency.
+- Display of loaded script for improved debugging and verification.
+
+### Fixed
+
+- Resolved target analysis issues, ensuring accurate process targeting.
+- Corrected function argument count errors for more robust script execution.
+- Fixed function selection by name to accurately identify and use the correct functions.
+
+## [1.0.2] - 2024-03-31
+
+### Added
+
+- Added support for interpreting and using symbols, enhancing analysis capabilities.
+
+### Changed
+
+- Optimized analysis logic during the hook process for increased efficiency.
+- Improved script generation process for more reliable and effective hooking.
+
+# [1.0.1] - 2024-03-31
+
+### Added
+
+- Introduced support for non-standard version handling, accommodating a wider range of target applications.
+
+## [1.0.0] - 2024-03-30
+
+### Added
+
+- Initial release of the project, laying the foundation for future enhancements and features.
+
+[1.0.4]: https://github.com/hyugogirubato/KeyDive/releases/tag/v1.0.4
+[1.0.3]: https://github.com/hyugogirubato/KeyDive/releases/tag/v1.0.3
+[1.0.2]: https://github.com/hyugogirubato/KeyDive/releases/tag/v1.0.2
+[1.0.1]: https://github.com/hyugogirubato/KeyDive/releases/tag/v1.0.1
+[1.0.0]: https://github.com/hyugogirubato/KeyDive/releases/tag/v1.0.0

README.md

@@ -3,7 +3,8 @@
 KeyDive is a sophisticated Python script designed for the precise extraction of Widevine L3 DRM (Digital Rights Management) keys from Android devices. This tool leverages the capabilities of the Widevine CDM (Content Decryption Module) to facilitate the recovery of DRM keys, enabling a deeper understanding and analysis of the Widevine L3 DRM implementation across various Android SDK versions.
 
 > [!IMPORTANT]
-> Support for Android 14+ (SDK > 33) is currently under development. Some features may not function as expected on these newer versions.
+> 
+> Support for Android 14+ (SDK > 33) require the use of functions extracted from Ghidra.
 
 ## Features
 
@@ -44,15 +45,17 @@ This sequence ensures that the DRM-protected content is active and ready for key
 ### Command-Line Options
 
 ```shell
-usage: keydive.py [-h] [--device DEVICE] [--functions FUNCTIONS]
+usage: keydive.py [-h] [-d DEVICE] [-f FUNCTIONS] [--force]
 
 Extract Widevine L3 keys from an Android device.
 
 options:
   -h, --help            show this help message and exit
-  --device DEVICE       Target Android device ID.
+  -d DEVICE, --device DEVICE
-  --functions FUNCTIONS
+                        Target Android device ID.
-                        Ghidra XML functions file.
+  -f FUNCTIONS, --functions FUNCTIONS
+                        Path to Ghidra XML functions file.
+  --force               Force using the default vendor (skipping analysis).
 
 ```
 

docs/version/18.0.0/libraries.log

@@ -1,19 +0,0 @@
-[
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=android.hardware.drm-service.widevine, base=0x5ac8c61f0000, size=2727936, path=/apex/com.google.android.widevine/bin/hw/android.hardware.drm-service.widevine)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=linker64, base=0x7a1e2ec4d000, size=290816, path=/system/bin/linker64)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libcrypto.so, base=0x7a1e29844000, size=1617920, path=/apex/com.google.android.widevine/lib64/libcrypto.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=liblog.so, base=0x7a1e2b2ad000, size=73728, path=/system/lib64/liblog.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libbinder_ndk.so, base=0x7a1e2e8c7000, size=118784, path=/system/lib64/libbinder_ndk.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libc.so, base=0x7a1e2820c000, size=5476352, path=/apex/com.android.runtime/lib64/bionic/libc.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libm.so, base=0x7a1e2d489000, size=278528, path=/apex/com.android.runtime/lib64/bionic/libm.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libdl.so, base=0x7a1e29820000, size=20480, path=/apex/com.android.runtime/lib64/bionic/libdl.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libc++.so, base=0x7a1e2b2e3000, size=749568, path=/system/lib64/libc++.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libbinder.so, base=0x7a1e2d512000, size=888832, path=/system/lib64/libbinder.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libutils.so, base=0x7a1e2b24b000, size=126976, path=/system/lib64/libutils.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libcutils.so, base=0x7a1e2878a000, size=102400, path=/system/lib64/libcutils.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libvndksupport.so, base=0x7a1e2d477000, size=16384, path=/system/lib64/libvndksupport.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libbase.so, base=0x7a1e2e860000, size=274432, path=/system/lib64/libbase.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libdl_android.so, base=0x7a1e287d1000, size=12288, path=/apex/com.android.runtime/lib64/bionic/libdl_android.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=libnetd_client.so, base=0x7a1c180e3000, size=45056, path=/system/lib64/libnetd_client.so)",
-  "Library(process=Process(pid=5399, name=\"android.hardware.drm-service.widevine\", parameters={}), name=linux-vdso.so.1, base=0x7fff16bf7000, size=4096, path=linux-vdso.so.1)"
-]

docs/version/18.0.0/paths.log

@@ -1,39 +0,0 @@
-/data_mirror/misc_de/null/0/apexdata/com.google.android.widevine
-/data_mirror/misc_ce/null/0/apexdata/com.google.android.widevine
-/vendor/apex/com.google.android.widevine.nonupdatable.apex
-/linkerconfig/com.google.android.widevine
-/linkerconfig/com.google.android.widevine/ld.config.txt
-/dev/f3o/.magisk/mirror/data/misc/apexdata/com.google.android.widevine
-/dev/f3o/.magisk/mirror/data/system/package_cache/d499bb796bfde9867e4d86a4c6309a64a05dfb0d/com.google.android.widevine.nonupdatable.apex-16--1978776024
-/dev/f3o/.magisk/mirror/data/misc_ce/0/apexdata/com.google.android.widevine
-/dev/f3o/.magisk/mirror/data/misc_de/0/apexdata/com.google.android.widevine
-/dev/f3o/.magisk/mirror/vendor/apex/com.google.android.widevine.nonupdatable.apex
-find: '/dev/f3o/.magisk/mirror/system_root': loop detected
-/data/misc/apexdata/com.google.android.widevine
-/data/system/package_cache/d499bb796bfde9867e4d86a4c6309a64a05dfb0d/com.google.android.widevine.nonupdatable.apex-16--1978776024
-/data/misc_ce/0/apexdata/com.google.android.widevine
-/data/misc_de/0/apexdata/com.google.android.widevine
-/apex/com.google.android.widevine
-/apex/com.google.android.widevine/lost+found
-/apex/com.google.android.widevine/bin
-/apex/com.google.android.widevine/bin/hw
-/apex/com.google.android.widevine/bin/hw/android.hardware.drm-service.widevine
-/apex/com.google.android.widevine/lib64
-/apex/com.google.android.widevine/lib64/libcrypto.so
-/apex/com.google.android.widevine/apex_manifest.pb
-/apex/com.google.android.widevine/etc
-/apex/com.google.android.widevine/etc/com.google.android.widevine.rc
-/apex/com.google.android.widevine/etc/vintf
-/apex/com.google.android.widevine/etc/vintf/com.google.android.widevine.xml
-/apex/com.google.android.widevine@340720000
-/apex/com.google.android.widevine@340720000/lost+found
-/apex/com.google.android.widevine@340720000/bin
-/apex/com.google.android.widevine@340720000/bin/hw
-/apex/com.google.android.widevine@340720000/bin/hw/android.hardware.drm-service.widevine
-/apex/com.google.android.widevine@340720000/lib64
-/apex/com.google.android.widevine@340720000/lib64/libcrypto.so
-/apex/com.google.android.widevine@340720000/apex_manifest.pb
-/apex/com.google.android.widevine@340720000/etc
-/apex/com.google.android.widevine@340720000/etc/com.google.android.widevine.rc
-/apex/com.google.android.widevine@340720000/etc/vintf
-/apex/com.google.android.widevine@340720000/etc/vintf/com.google.android.widevine.xml

extractor/__init__.py

@@ -1,4 +1,4 @@
 from .cdm import *
 from .vendor import *
 
-__version__ = '1.0.3'
+__version__ = '1.0.4'

extractor/cdm.py

@@ -6,7 +6,6 @@ from pathlib import Path
 
 import xmltodict
 import frida
-from _frida import Process
 from frida.core import Device, Session, Script
 from Cryptodome.PublicKey import RSA
 
@@ -28,23 +27,35 @@ class Cdm:
         # Add more as needed for different versions.
     }
 
-    def __init__(self, device: str = None, functions: Path = None):
+    def __init__(self, device: str = None, functions: Path = None, force: bool = False):
         self.logger = logging.getLogger('Cdm')
+        self.functions = functions
         self.running = True
         self.keys = {}
+        # Select device based on provided ID or default to the first USB device.
         self.device: Device = frida.get_device(id=device, timeout=5) if device else frida.get_usb_device(timeout=5)
         self.logger.info('Device: %s (%s)', self.device.name, self.device.id)
 
-        # Fetch and log device properties
+        # Obtain device properties
         self.properties = self._fetch_device_properties()
+
         self.sdk_api = self.properties['ro.build.version.sdk']
         self.logger.info('SDK API: %s', self.sdk_api)
         self.logger.info('ABI CPU: %s', self.properties['ro.product.cpu.abi'])
 
-        # Determine vendor based on SDK API
+        # Load the hook scrip
-        self.script = self._prepare_hook_script(functions)
+        self.script = self._prepare_hook_script()
-        self.logger.info('Successfully loaded script')
+        self.logger.info('Script loaded successfully')
-        self.vendor = self._prepare_vendor()
+
+        # Determine vendor based on device SDK API
+        vendor_api = self._prepare_vendor_api(force=force)
+        self.vendor = Vendor.from_sdk_api(vendor_api)
+
+        # Update script for specific vendor API, if necessary
+        if vendor_api != self.sdk_api:
+            self.sdk_api = vendor_api
+            self.script = self._prepare_hook_script()
+            self.logger.info('Script updated for vendor API')
 
     def _fetch_device_properties(self) -> dict:
         """
@@ -65,76 +76,112 @@ class Cdm:
                 properties[key] = value
         return properties
 
-    def _prepare_hook_script(self, path: Path) -> str:
+    def _prepare_hook_script(self) -> str:
         """
-        Prepares and returns the hook script by replacing placeholders with actual values, including
+        Prepares the Frida hook script, injecting dynamic content like SDK API and selected functions.
-        SDK API version and selected functions from a given XML file.
         """
-        selected = {}
-        if path:
-            # Verify symbols file path
-            if not path.is_file():
-                raise FileNotFoundError('Symbols file not found')
-
-            try:
-                # Parse the XML file
-                program = xmltodict.parse(path.read_bytes())['PROGRAM']
-                addr_base = int(program['@IMAGE_BASE'], 16)
-                functions = program['FUNCTIONS']['FUNCTION']
-
-                # Find a target function from a predefined list
-                target = next((f['@NAME'] for f in functions if f['@NAME'] in self.OEM_CRYPTO_API), None)
-
-                # Extract relevant functions
-                for func in functions:
-                    name = func['@NAME']
-                    args = len(func.get('REGISTER_VAR', []))
-
-                    # Add function if it matches specific criteria
-                    if name not in selected and (
-                            name == target
-                            or any(keyword in name for keyword in ['UsePrivacyMode', 'PrepareKeyRequest'])
-                            or (not target and re.match(r'^[a-z]+$', name) and args >= 6)
-                    ):
-                        selected[name] = {'name': name, 'address': hex(int(func['@ENTRY_POINT'], 16) - addr_base)}
-            except Exception:
-                raise ValueError('Failed to extract functions from Ghidra')
-
-        # Read and prepare the hook script content
         content = SCRIPT_PATH.read_text(encoding='utf-8')
-        # Replace placeholders with actual values
+        selected = self._select_functions() if self.functions else {}
-        content = content.replace('${SDK_API}', str(self.sdk_api))
+
-        content = content.replace('${OEM_CRYPTO_API}', json.dumps(list(self.OEM_CRYPTO_API)))
+        # Replace placeholders in script template
-        content = content.replace('${SYMBOLS}', json.dumps(list(selected.values())))
+        replacements = {
+            '${SDK_API}': str(self.sdk_api),
+            '${OEM_CRYPTO_API}': json.dumps(list(self.OEM_CRYPTO_API)),
+            '${SYMBOLS}': json.dumps(list(selected.values())),
+        }
+
+        for placeholder, real_value in replacements.items():
+            content = content.replace(placeholder, real_value)
 
         return content
 
-    def _prepare_vendor(self) -> Vendor:
+    def _select_functions(self) -> dict:
         """
-        Prepares and selects the most compatible vendor version based on the device's processes.
+        Parses the provided XML functions file to select relevant functions.
         """
+        if not self.functions.is_file():
+            raise FileNotFoundError('Functions file not found')
+
+        try:
+            program = xmltodict.parse(self.functions.read_bytes())['PROGRAM']
+            addr_base = int(program['@IMAGE_BASE'], 16)
+            functions = program['FUNCTIONS']['FUNCTION']
+
+            # Find a target function from a predefined list
+            target = next((f['@NAME'] for f in functions if f['@NAME'] in self.OEM_CRYPTO_API), None)
+
+            # Extract relevant functions
+            selected = {}
+            for func in functions:
+                name = func['@NAME']
+                args = len(func.get('REGISTER_VAR', []))
+
+                # Add function if it matches specific criteria
+                if name not in selected and (
+                        name == target
+                        or any(keyword in name for keyword in ['UsePrivacyMode', 'PrepareKeyRequest'])
+                        or (not target and re.match(r'^[a-z]+$', name) and args >= 6)
+                ):
+                    selected[name] = {'name': name, 'address': hex(int(func['@ENTRY_POINT'], 16) - addr_base)}
+            return selected
+        except Exception:
+            pass
+        raise ValueError('Failed to extract functions from Ghidra')
+
+    def enumerate_processes(self) -> dict:
+        """
+        Lists processes running on the device, returning a mapping of process names to PIDs.
+        """
+        # https://github.com/frida/frida/issues/2593
+        # Iterate through lines starting from the second line (skipping header)
+        processes = {}
+        for line in subprocess.getoutput(f'adb -s "{self.device.id}" shell ps').splitlines()[1:]:
+            try:
+                line = line.split()  # USER,PID,PPID,VSZ,RSS,WCHAN,ADDR,S,NAME
+                name = ' '.join(line[8:]).strip()
+                name = name if name.startswith('[') else Path(name).name
+                processes[name] = int(line[1])
+            except Exception:
+                pass
+
+        return processes
+
+    def _prepare_vendor_api(self, force: bool = False) -> int:
+        """
+        Determines the most compatible vendor API version based on device processes.
+        """
+        if force:
+            self.logger.warning('Using default vendor due to force flag')
+            return self.sdk_api
+
+        # Check if forcing is not enabled and enumerate processes
         details: [int] = []
-        for p in self.device.enumerate_processes():
+        processes = self.enumerate_processes()
-            for k, v in Vendor.SDK_VERSIONS.items():
+        for k, v in Vendor.SDK_VERSIONS.items():
-                if p.name == v[2]:
+            pid = processes.get(v[2])
-                    session: Session = self.device.attach(p.name)
+            if pid:
-                    script: Script = session.create_script(self.script)
+                self.logger.debug('Analysing... (%s)', v[2])
-                    script.load()
+                session: Session = self.device.attach(pid)
-                    if script.exports_sync.getlibrary(v[3]):
+                script: Script = session.create_script(self.script)
-                        details.append(k)
+                script.load()
-                    session.detach()
+                if script.exports_sync.getlibrary(v[3]):
+                    details.append(k)
+                session.detach()
 
-        if not details:
+        # If no compatible versions found
-            return Vendor.from_sdk_api(self.sdk_api)
+        if details:
+            # Find the closest SDK version to the current one, preferring lower matches in case of a tie.
+            sdk_api = min(details, key=lambda x: abs(x - self.sdk_api))
 
-        # Find the closest SDK version to the current one, preferring lower matches in case of a tie.
+            # Adjust SDK version if it exceeds the maximum supported version
-        sdk_api = min(details, key=lambda x: abs(x - self.sdk_api))
+            if sdk_api == Vendor.SDK_MAX and self.sdk_api > Vendor.SDK_MAX:
-        if sdk_api == Vendor.SDK_MAX and self.sdk_api > Vendor.SDK_MAX:
+                sdk_api = self.sdk_api
-            sdk_api = self.sdk_api
+            elif sdk_api != self.sdk_api:
-        elif sdk_api != self.sdk_api:
+                self.logger.warning('Using non-default Widevine version for SDK %s', sdk_api)
-            self.logger.warning('Non-default Widevine version for SDK %s', sdk_api)
 
-        return Vendor.from_sdk_api(sdk_api)
+            return sdk_api
+
+        raise EnvironmentError('Unable to detect Widevine, see: https://github.com/hyugogirubato/KeyDive/blob/main/docs/PACKAGE.md#drm-info')
 
     def _process_message(self, message: dict, data: bytes) -> None:
         """
@@ -206,11 +253,11 @@ class Cdm:
         else:
             self.logger.warning('Failed to intercept the private key')
 
-    def hook_process(self, process: Process) -> bool:
+    def hook_process(self, pid: int) -> bool:
         """
         Hooks into the specified process to intercept DRM keys.
         """
-        session: Session = self.device.attach(process.name)
+        session: Session = self.device.attach(pid)
         script: Script = session.create_script(self.script)
         script.on('message', self._process_message)
         script.load()
@@ -218,5 +265,13 @@ class Cdm:
         library_info = script.exports_sync.getlibrary(self.vendor.library)
         if library_info:
             self.logger.info('Library: %s (%s)', library_info['name'], library_info['path'])
+
+            # Check if Ghidra XML functions loaded
+            if self.sdk_api > 33:
+                if not self.functions:
+                    raise AttributeError('For SDK API > 33, specifying "functions" is required, see: https://github.com/hyugogirubato/KeyDive/blob/main/docs/FUNCTIONS.md')
+            elif self.functions:
+                self.logger.warning('The "functions" attribute is deprecated for SDK API < 34')
+
             return script.exports_sync.hooklibrary(library_info['name'])
         return False

extractor/script.js

@@ -82,7 +82,6 @@ const hookLibrary = (name) => {
             'name': symbol.name,
             'address': ptr(parseInt(symbol.address, 16) + parseInt(library.base, 16))
         }));
-        print(Level.INFO, 'Successfully imported functions');
     } else {
         functions = [...library.enumerateExports(), ...library.enumerateImports()];
         target = functions.find(func => OEM_CRYPTO_API.includes(func.name));

keydive.py

@@ -4,7 +4,6 @@ import subprocess
 import time
 
 import coloredlogs
-from _frida import Process
 from pathlib import Path
 
 import extractor
@@ -21,29 +20,31 @@ if __name__ == '__main__':
     # Parse command line arguments for device ID
     parser = argparse.ArgumentParser(description='Extract Widevine L3 keys from an Android device.')
     parser.add_argument('-d', '--device', required=False, type=str, help='Target Android device ID.')
-    parser.add_argument('-f', '--functions', required=False, type=Path, help='Ghidra XML functions file.')
+    parser.add_argument('-f', '--functions', required=False, type=Path, help='Path to Ghidra XML functions file.')
+    parser.add_argument('--force', required=False, action='store_true', help='Force using the default vendor (skipping analysis).')
+
     args = parser.parse_args()
 
     try:
         logger.info('Version: %s', extractor.__version__)
 
-        # Start ADB server
+        # Ensure the ADB server is running
         exitcode, _ = subprocess.getstatusoutput('adb start-server')
         if exitcode != 0:
-            raise EnvironmentError('ADB is not recognized as an environment variable')
+            raise EnvironmentError('ADB is not recognized as an environment variable, see https://github.com/hyugogirubato/KeyDive/blob/main/docs/PACKAGE.md#adb-android-debug-bridge')
 
-        # Initialize CDM handler with given device
+        # Initialize the CDM handler with the specified or default device
-        cdm = Cdm(device=args.device, functions=args.functions)
+        cdm = Cdm(device=args.device, functions=args.functions, force=args.force)
 
-        # Find Widevine process on the device
+        # Attempt to locate and identify the Widevine process on the target device
-        process: Process = next((p for p in cdm.device.enumerate_processes() if cdm.vendor.process == p.name), None)
+        pid = cdm.enumerate_processes().get(cdm.vendor.process)
-        if not process:
+        if not pid:
-            raise Exception('Failed to find Widevine process')
+            raise EnvironmentError('Widevine process not found on the device')
-        logger.info('Process: %s (%s)', process.pid, process.name)
+        logger.info('Process: %s (%s)', pid, cdm.vendor.process)
 
-        # Hook into the process to extract DRM keys
+        # Hook into the identified process for DRM key extraction
-        if not cdm.hook_process(process):
+        if not cdm.hook_process(pid=pid):
-            raise Exception('Failed to hook into the process')
+            raise Exception('Failed to hook into the Widevine process')
         logger.info('Successfully hooked. To test, play a DRM-protected video: https://bitmovin.com/demos/drm')
 
         # Keep script running while extracting keys