[beauregardhenry]

Summary

Hardens request and command handling, fixes several bugs in device handling and data deletion, adds CI with lint ratchets, and splits evidence_collection.py into smaller modules. The test suite goes from none to 398 tests.

This is a large PR. Each section below maps to a group of commits, so it can be reviewed by theme. I can split it into separate PRs if you prefer.

Motivation

Sherloc handles sensitive client data in a clinic setting, so the failure modes matter more than in a typical Flask app. Three problems stood out:

• "Delete client data" removed files but left client notes and scan rows in fieldstudy.db.
• Device serials and app ids reached shell commands without validation.
• Several routes and buttons were broken or unsafe (/view_results, "Close App", uninstall on multi-device setups).

Changes

Security

• Validate device serials and app ids before they reach a shell command (sherloc/inputcheck.py), and fix the quoting in the Android and iOS uninstall commands.
• Reject requests with a foreign Host, a cross-origin Origin on state-changing methods, or a non-same-origin Sec-Fetch-Site. The app binds to 127.0.0.1 by default. SHERLOC_HOST and SHERLOC_ALLOWED_HOSTS change this.
• Untrack pii.key and flask.secret. Keys are generated on first use with 0600 permissions. A key file that still holds a previously published value is replaced.
• Escape user text in the PDF printout and turn off local file access in wkhtmltopdf.
• Reduce app-store descriptions to a short allow-list of formatting tags before rendering.
• Delete, kill and delete-app routes are POST-only. The buttons are now forms.
• Limit screenshot paths to the screenshots directory.
• Request log lines show the route pattern, not the URL. SQL echo is off by default (SHERLOC_SQL_ECHO=1 turns it on).

Bug fixes

• "Delete client data" now also empties the database, overwrites deleted content and compacts the file. It also no longer stops at the first missing folder.
• Android and iOS uninstall target the scanned device (adb -s, ideviceinstaller --udid). delete_app checks that the serial matches the scan.
• iOS screenshots work when the install path has a space. The pymobiledevice3 tunnel is stopped after each screenshot.
• "Close App and End Session" works again. It relied on werkzeug.server.shutdown, which Werkzeug 2.1 removed.
• /view_results no longer raises NameError. Dead code that referenced undefined names is removed.
• python main.py test now enters test mode. The argument is applied before config is imported.
• A missing blocklist file now exits with an error instead of status 0.
• 30 mutable default arguments, 15 invalid escape sequences and one bare except are fixed.

Refactors (no behavior change)

• config no longer creates files or folders on import.
• Removed the duplicate run_command. The remaining one is run_capture.
• Split evidence_collection.py (1,942 lines) into evidence_choices.py, evidence_model.py and evidence_forms.py. evidence_collection re-exports every public name, and a test pins that API.

CI

• tests.yml runs pytest on pushes and pull requests.
• tests/test_lint_ratchet.py fails on new undefined names, invalid escapes, mutable defaults, bare except, shell=True and eval. Existing findings are recorded in tests/lint_baseline.json (2 remain).
• super-linter checks changed files for serious flake8 errors and executable bits. The style linters were failing on about 170 existing findings.
• All GitHub Actions are pinned to commit hashes, with a test that enforces it.
• The stalkerware-indicators workflow had a duplicate token: key that made IOC_UPDATE_KEY ignored. It no longer tries to open a PR on pull request runs.

Breaking changes

• Existing installs get new keys on pull. Stored HSN_ device identifiers will change. The old key values remain in git history.
• Delete and kill endpoints no longer accept GET. Anything that called them with a link or fetch GET must use POST.
• Sherloc listens on 127.0.0.1 by default. Anyone serving it to other machines must set SHERLOC_HOST and SHERLOC_ALLOWED_HOSTS.
• The serial validator is an allow-list. A real device with an unusual serial (an IPv6 adb target, for example) may be rejected.

Testing

• pytest from the repo root: 398 passed, 1 skipped (needs exiftool).
• Bug fixes and security changes were written test-first and confirmed failing before the fix.
• I did not click through the UI in a browser, and I did not test a real iOS uninstall. --udid is taken from the ideviceinstaller man page. Please check both if you have a device.
• The iOS tunnel device selection (start-tunnel) is unverified.

Checklist

• Tests added or updated
• CHANGELOG.md updated under [Unreleased]
• CI passes (pytest, run-lint)
• No new lint findings against the baseline
• Manual browser check of the delete buttons and "Close App" (maintainer or reviewer with a device)

[sophiestephenson]

Hi @beauregardhenry. Thank you for submitting this PR. I see that Claude has made most of these commits, which raises warning signs for me. Before I review the changes, I would like to know a little more about why you are submitting this PR and what motivated the specific changes that have been made. Please describe this in a comment. Thank you.

[beauregardhenry]

Fair. Please disregard and think no more about it.

I intended to contribute to Sherloc, a separate project with a different aim.

Love the work that you've done.

Please accept my apology for making a mistake.

Sincerely,
Beau

[beauregardhenry]

I saw that your project was initially based on ISDi. I have looked at ISDi and created my own fork which goes much further than the original project. I believe your project would benefit from a CI/CD pipeline, hardening measures, refactoring, test coverage, and test ratchets. That was the intent. The area where I live needs projects like Sherloc, and I wanted to contribute meaningfully. I may have gone about it wrongly. I will continue working on my fork of your project. If at any time you would like to consider merging some of what I do, please let me know and I would be happy to work together. I do not remember the acronym at the moment, but technology enabled abuse needs to end. I would enjoy contributing to any project that uses technology to catch abusers and provide evidence that strengthens cases for those who stand up.