create module test harness

admin/docs/testing/create_module_test_cases.md

@@ -0,0 +1,107 @@
+# Creating Module Test Cases
+
+This document describes the process of creating test cases for LEAPP modules using the `make_test_data.py` script.
+
+## Overview
+
+The `make_test_data.py` script is designed to generate test data for LEAPP modules. It processes input files (zip, tar, or tar.gz) to extract relevant files based on the module's artifact patterns and creates structured test cases.
+
+## Usage
+
+To create test cases for a module, use the following command:
+
+```python 
+make_test_data.py <module_name> <case_number> <input_file>
+- `<module_name>`: Name of the module (e.g., keyboard or keyboard.py)
+- `<case_number>`: Case number for the test data
+- `<input_file>`: Path to the input file (zip, tar, or tar.gz)
+```
+## Process
+
+1. The script imports the specified module and retrieves artifact information.
+2. It creates or updates a JSON file with test case metadata.
+3. The script processes the input archive file, searching for files matching the artifact patterns.
+4. For each artifact, it creates a zip file containing the matching files.
+
+5. The JSON file is updated with information about the created test data.
+
+## Output
+
+The script generates the following outputs:
+
+1. A JSON file containing test case metadata.
+    - `admin/test/cases/testdata.<module_name>.json`
+2. Zip files for each artifact, containing the relevant test data files.
+    - `admin/test/cases/data/testdata.<module_name>.<artifact_name>.<case_number>.zip`
+
+## Example
+
+```python 
+make_test_data.py keyboard 001 /path/to/input/data.zip
+```
+
+This command will create test data for the keyboard module, using case number 001 and the specified input file.
+
+## Notes
+
+- The script supports zip, tar, and tar.gz input files.
+- Test data is stored in the `admin/test/cases/data` directory.
+- JSON metadata files are stored in the `admin/test/cases` directory.
+- Always review and update the generated JSON file with additional test case details as needed.
+
+## Test Case JSON File Structure
+
+The script generates a JSON file (e.g., `testdata.<module_name>.json`) that contains metadata and information about the test cases. This file is crucial for running tests and maintaining test data. Here's an explanation of its structure:
+
+```json
+{
+  "case001": {
+    "description": "",
+    "maker": "",
+    "make_data": {
+      "input_data_path": "/path/to/input/data.tar.gz",
+      "os": "macOS-15.0-x86_64-i386-64bit",
+      "timestamp": "2024-10-14T10:17:49.432528"
+    },
+    "artifacts": {
+      "artifact_name": {
+        "search_patterns": [
+          "*/path/to/artifact/files"
+        ],
+        "file_count": 1,
+        "expected_output": {
+          "headers": [],
+          "data": []
+        }
+      }
+    }
+  }
+}
+```
+
+- `case001`: A unique identifier for each test case.
+  - `description`: A brief description of the test case (to be filled in manually).
+  - `maker`: The person who created the test case (to be filled in manually).
+  - `make_data`: Information about the test case creation process.
+    - `input_data_path`: The path to the input file used to create the test case.
+    - `os`: The operating system on which the test case was created.
+    - `timestamp`: The date and time when the test case was created.
+  - `artifacts`: A dictionary of artifacts tested in this case.
+    - `artifact_name`: The name of the artifact (e.g., "get_keyboard_lexicon").
+      - `search_patterns`: The file patterns used to find relevant files.
+      - `file_count`: The number of files found matching the search patterns.
+      - `expected_output`: The expected results of the artifact extraction.
+        - `headers`: Column headers for the expected output (to be filled in manually).
+        - `data`: Expected data rows (to be filled in manually).
+
+Multiple test cases (e.g., "case001", "case002") can be included in a single JSON file.
+
+## Updating the JSON File
+
+After creating test cases, it's important to manually update the JSON file with the following information:
+
+1. Add a meaningful description for each test case.
+2. Include the name of the person who created the test case.
+3. Fill in the expected output headers and data for each artifact.
+
+This information is crucial for validating test results and ensuring the accuracy of the artifact extraction process.

admin/docs/testing/readme.md

@@ -0,0 +1,42 @@
+# LEAPP Testing Documentation
+
+This documentation provides an overview of the testing processes used in the LEAPP project, with a current focus on module testing.
+
+## Module Testing
+
+Module testing is a crucial part of ensuring the reliability and accuracy of LEAPP's artifact extraction and analysis capabilities. Our module testing process involves two main steps:
+
+1. [Creating Module Test Cases](create_module_test_cases.md)
+2. [Testing Modules](testing_modules.md)
+
+### Creating Module Test Cases
+
+We use a custom script to generate test cases for each module. This process involves:
+
+- Extracting relevant files from input data (zip, tar, or tar.gz files)
+- Creating JSON files with test case metadata
+- Generating zip files containing test data for each artifact
+
+For more details, see the [Creating Module Test Cases](create_module_test_cases.md) documentation.
+
+### Testing Modules
+
+Once test cases are created, we use another script to run tests on the modules. This process includes:
+
+- Selecting specific modules, artifacts, and test cases to run
+- Executing the module's artifact extraction function
+- Comparing the results with expected output
+- Generating test result files
+
+For more information, refer to the [Testing Modules](testing_modules.md) documentation.
+
+## Future Expansions
+
+As the LEAPP project grows, we plan to expand our testing documentation to cover:
+
+- Unit testing
+- Integration testing
+- Performance testing
+- Continuous Integration (CI) processes
+
+

admin/docs/testing/testing_modules.md

@@ -0,0 +1,55 @@
+# Testing Modules
+
+This document describes the process of testing LEAPP modules using the `test_module.py` script.
+
+## Overview
+
+The `test_module.py` script is designed to run tests on LEAPP modules using previously created test cases. It processes the test data, executes the module's artifact extraction function, and generates test results.
+
+## Usage
+
+To test a module, use the following command:
+
+``` python 
+python test_module.py <module_name> [artifact_name] [case_number]
+
+
+- `<module_name>`: Name of the module to test
+- `[artifact_name]`: (Optional) Name of the specific artifact to test (or 'all')
+- `[case_number]`: (Optional) Specific case number to test (or 'all')
+```
+- If `artifact_name` or `case_number` are not provided, the script will prompt you to select from available options or run all.
+- Provide `all` on the command line to run all test cases and artifacts.
+
+## Process
+
+1. The script loads test cases for the specified module.
+2. It allows selection of specific artifacts and test cases to run.
+3. For each selected test case and artifact:
+   a. The script extracts test data from the corresponding zip file.
+   b. It executes the module's artifact extraction function.
+   c. The results are collected and formatted.
+4. Test results are saved as JSON files.
+
+## Output
+
+The script generates JSON files containing test results, including:
+
+- Metadata about the test run (module name, artifact name, case number, etc.)
+- Execution time and performance metrics
+- The extracted data (headers and rows)
+
+Output files are stored in the `admin/test/results` directory.
+
+## Example
+
+```python test_module.py keyboard```
+
+
+This command will start the testing process for the keyboard module, prompting you to select specific artifacts and test cases.
+
+## Notes
+
+- The script uses test data created by the `make_test_data.py` script.
+- Test results include information about the last Git commit for the tested module.
+- You can run tests for all artifacts and all test cases by selecting 'all' when prompted.

admin/test/cases/testdata.keyboard.json

@@ -0,0 +1,50 @@
+{
+  "case001": {
+    "description": "",
+    "maker": "",
+    "make_data": {
+      "input_data_path": "/Users/jameshabben/Documents/ios backup play/Josh/iOS_15_Public_Image.tar.gz",
+      "os": "macOS-15.0-x86_64-i386-64bit",
+      "timestamp": "2024-10-14T11:54:02.029434",
+      "last_commit": {
+        "hash": "25ee1e18a1ff21d062595753ba9b33bfc73df248",
+        "author_name": "stark4n6",
+        "author_email": "48143894+stark4n6@users.noreply.github.com",
+        "date": "2024-03-08T10:29:04-05:00",
+        "message": "Update keyboard.py"
+      }
+    },
+    "artifacts": {
+      "get_keyboard_lexicon": {
+        "search_patterns": [
+          "*/mobile/Library/Keyboard/*-dynamic.lm/dynamic-lexicon.dat"
+        ],
+        "file_count": 1,
+        "expected_output": {
+          "headers": [],
+          "data": []
+        }
+      },
+      "get_keyboard_app_usage": {
+        "search_patterns": [
+          "*/mobile/Library/Keyboard/app_usage_database.plist"
+        ],
+        "file_count": 1,
+        "expected_output": {
+          "headers": [],
+          "data": []
+        }
+      },
+      "get_keyboard_usage_stats": {
+        "search_patterns": [
+          "*/mobile/Library/Keyboard/user_model_database.sqlite*"
+        ],
+        "file_count": 3,
+        "expected_output": {
+          "headers": [],
+          "data": []
+        }
+      }
+    }
+  }
+}

admin/test/results/keyboard_get_keyboard_lexicon_case001_20241014182516.json

@@ -0,0 +1,25 @@
+{
+  "metadata": {
+    "module_name": "keyboard",
+    "artifact_name": "Keyboard Dynamic Lexicon",
+    "function_name": "get_keyboard_lexicon",
+    "case_number": "case001",
+    "number_of_columns": 2,
+    "number_of_rows": 1,
+    "total_data_size_bytes": 2961,
+    "input_zip_path": "admin/test/cases/data/testdata.keyboard.get_keyboard_lexicon.case001.zip",
+    "start_time": "2024-10-14T18:25:16.578197+00:00",
+    "end_time": "2024-10-14T18:25:16.583042+00:00",
+    "run_time_seconds": 0.0008459091186523438
+  },
+  "headers": [
+    "Found Strings",
+    "File Location"
+  ],
+  "data": [
+    [
+      "3rd,N(%O4fa3,about,N(%Oacb4,accident,actually,ads,N(%Oaf1a,afternoon,after,against,again,agreed,alarm,N(%Alarm,all,also,amazed,Android,and,another,any,apparently,appear,apps,app,6ap,are,around,attached,N(d,Attachment,attachment,audio,Austin,avatar,awesome,backwards,back,bad,N'Bandit,basic,battery,Beach,N(%Obeee,been,before,believe,+Below,best,better,bigger,big,binge,bite,bit,both,breakfast,burned,burner,but,calling,calls,call,canceled,cannot,can't,can,catching,charged,chat,Chinese,chose,close,clue,comes,confusing,cooked,correct,could,covered,cracked,crazy,create,creating,crowded,daily,N(%Daily,data,day,deal,decided,+Deck,definitely,deleted,delete,devices,didn't,did,dinner,DM's,DMs,doesn't,does,doing,N(,gDominant,N(,iDominion,done,don't,download,down,N(,cDo,drained,dropped,N'DS9,dying,eat,edit,emojis,episode,N(%Event,everyone,everything,excuse,experienced,experiencing,extra,N(%Of5c4,FaceTime,fair,fam,favorite,features,few,figure,find,finish,first,forgot,for,from,general,generated,generate,generating,getting,get,give,glad,going,gone,good,Google,got,grabbed,great,group,_Group,guess,Hal,handled,hangs,hang,happened,happens,happen,hard,has,have,head,heard,hearing,heck,hello,here,hey,hide,Hiw,horrendous,horrible,hours,hour,how's,how,huddle,hustle,N'Ibaudio,I'll,ill,N'Images,image,N(%qImage,I'm,incoming,initially,_Ios15,iOS,iPhone,$Is,it's,I've,June,just,keep,keyboard,kids,N'Kik,Kim,knows,know,lame,last,later,leaning,learn,let's,let,like,lines,list,location,lol,Lol,long,looks,look,lot,lunch,machine,main,make,many,man,may,kma,meetings,meeting,N(d7Memes,messages,message,messaging,4miening,mine,minutes,minute,momentarily,moment,more,morning,mostly,move,much,N'Murica,myself,native,needs,need,never,new,next,nice,nope,note,N(,Note,nothing,noticed,notifications,not,now,number,odd,off,old,ones,one,only,honl,open,ouch,our,out,overdue,over,own,pain,party,part,pay,people,percent,phone,picture,N(%]Pic,pic,plans,please,plot,point,poor,portability,present,private,probably,quick,quite,raining,recall,received,regular,relax,removed,remove,reply,retiring,room,Room,run,sailboat,same,saved,saving,says,season,secure,seems,seem,see,sending,send,sent,server,service,setting,set,shall,should,shows,show,sidetracked,since,sir,sitting,slightly,slowly,slow,something,some,sooo,sorry,sounds,N'1Space,speaking,standby,started,start,still,storage,stuck,stuff,sucks,sure,switch,takeout,N'takeou,takes,tell,testing,N'Test,test,texts,thank,that's,that,them,then,there's,there,these,they,the,N(,iThe,things,thing,think,this,thought,though,through,time,today,too,total,towards,trashed,ytrashws,trashy,truck,true,truth,try,turn,two,typo,underrated,unfortunately,unsend,use,video,Voice,waffles,waffle,want,wasn't,was,watched,watching,week,welcome,well,we'll,we've,what's,\nb\t,what,when,which,N'Whic,who,why,will,winding,window,with,won't,would,Wrightsville,N(%wss,N(%]w,yay,yeah,yea,yep,Yes,yet,you'd,N(%IYou'reI,1You're,you're,your,you",
+      "en-dynamic.lm/dynamic-lexicon.dat"
+    ]
+  ]
+}

admin/test/results/keyboard_get_keyboard_lexicon_case001_20241014183005.json

@@ -0,0 +1,32 @@
+{
+  "metadata": {
+    "module_name": "keyboard",
+    "artifact_name": "Keyboard Dynamic Lexicon",
+    "function_name": "get_keyboard_lexicon",
+    "case_number": "case001",
+    "number_of_columns": 2,
+    "number_of_rows": 1,
+    "total_data_size_bytes": 2961,
+    "input_zip_path": "admin/test/cases/data/testdata.keyboard.get_keyboard_lexicon.case001.zip",
+    "start_time": "2024-10-14T18:30:05.265827+00:00",
+    "end_time": "2024-10-14T18:30:05.393523+00:00",
+    "run_time_seconds": 0.0005900859832763672,
+    "last_commit": {
+      "hash": "25ee1e18a1ff21d062595753ba9b33bfc73df248",
+      "author_name": "stark4n6",
+      "author_email": "48143894+stark4n6@users.noreply.github.com",
+      "date": "2024-03-08T10:29:04-05:00",
+      "message": "Update keyboard.py"
+    }
+  },
+  "headers": [
+    "Found Strings",
+    "File Location"
+  ],
+  "data": [
+    [
+      "3rd,N(%O4fa3,about,N(%Oacb4,accident,actually,ads,N(%Oaf1a,afternoon,after,against,again,agreed,alarm,N(%Alarm,all,also,amazed,Android,and,another,any,apparently,appear,apps,app,6ap,are,around,attached,N(d,Attachment,attachment,audio,Austin,avatar,awesome,backwards,back,bad,N'Bandit,basic,battery,Beach,N(%Obeee,been,before,believe,+Below,best,better,bigger,big,binge,bite,bit,both,breakfast,burned,burner,but,calling,calls,call,canceled,cannot,can't,can,catching,charged,chat,Chinese,chose,close,clue,comes,confusing,cooked,correct,could,covered,cracked,crazy,create,creating,crowded,daily,N(%Daily,data,day,deal,decided,+Deck,definitely,deleted,delete,devices,didn't,did,dinner,DM's,DMs,doesn't,does,doing,N(,gDominant,N(,iDominion,done,don't,download,down,N(,cDo,drained,dropped,N'DS9,dying,eat,edit,emojis,episode,N(%Event,everyone,everything,excuse,experienced,experiencing,extra,N(%Of5c4,FaceTime,fair,fam,favorite,features,few,figure,find,finish,first,forgot,for,from,general,generated,generate,generating,getting,get,give,glad,going,gone,good,Google,got,grabbed,great,group,_Group,guess,Hal,handled,hangs,hang,happened,happens,happen,hard,has,have,head,heard,hearing,heck,hello,here,hey,hide,Hiw,horrendous,horrible,hours,hour,how's,how,huddle,hustle,N'Ibaudio,I'll,ill,N'Images,image,N(%qImage,I'm,incoming,initially,_Ios15,iOS,iPhone,$Is,it's,I've,June,just,keep,keyboard,kids,N'Kik,Kim,knows,know,lame,last,later,leaning,learn,let's,let,like,lines,list,location,lol,Lol,long,looks,look,lot,lunch,machine,main,make,many,man,may,kma,meetings,meeting,N(d7Memes,messages,message,messaging,4miening,mine,minutes,minute,momentarily,moment,more,morning,mostly,move,much,N'Murica,myself,native,needs,need,never,new,next,nice,nope,note,N(,Note,nothing,noticed,notifications,not,now,number,odd,off,old,ones,one,only,honl,open,ouch,our,out,overdue,over,own,pain,party,part,pay,people,percent,phone,picture,N(%]Pic,pic,plans,please,plot,point,poor,portability,present,private,probably,quick,quite,raining,recall,received,regular,relax,removed,remove,reply,retiring,room,Room,run,sailboat,same,saved,saving,says,season,secure,seems,seem,see,sending,send,sent,server,service,setting,set,shall,should,shows,show,sidetracked,since,sir,sitting,slightly,slowly,slow,something,some,sooo,sorry,sounds,N'1Space,speaking,standby,started,start,still,storage,stuck,stuff,sucks,sure,switch,takeout,N'takeou,takes,tell,testing,N'Test,test,texts,thank,that's,that,them,then,there's,there,these,they,the,N(,iThe,things,thing,think,this,thought,though,through,time,today,too,total,towards,trashed,ytrashws,trashy,truck,true,truth,try,turn,two,typo,underrated,unfortunately,unsend,use,video,Voice,waffles,waffle,want,wasn't,was,watched,watching,week,welcome,well,we'll,we've,what's,\nb\t,what,when,which,N'Whic,who,why,will,winding,window,with,won't,would,Wrightsville,N(%wss,N(%]w,yay,yeah,yea,yep,Yes,yet,you'd,N(%IYou'reI,1You're,you're,your,you",
+      "en-dynamic.lm/dynamic-lexicon.dat"
+    ]
+  ]
+}