Skip to content

Auto-translated map_item updates from 4CAT: xiaohongshu - #103

Merged
dale-wahl merged 3 commits into
masterfrom
auto/4cat-map-item-sync-xiaohongshu
Oct 21, 2026
Merged

dale-wahl merged 3 commits into
masterfrom
auto/4cat-map-item-sync-xiaohongshu

Conversation

@4cat-to-zeeschuimer-automation-pr

Copy link
Copy Markdown
Contributor

🤖 This PR was auto-generated by the 4CAT map_item sync workflow. The JavaScript was produced by an LLM and requires human review before merging — including manual fixes for any lint warnings flagged below.

Generation parameters

  • Model: gpt-oss-120b (provider: litellm)
  • Total LLM time: 35.2s
  • Trigger: push of 451220d to 4CAT master (module: xiaohongshu).

Summary

  • ✅ 1 translated
  • ⚠️ 1 translated with lint warnings (require manual fix)
  • ❌ 0 failed
  • ❔ 0 skipped
Datasource Module Time Warnings
datasources/xiaohongshu/search_rednote.py modules/rednote.js 35.19s ⚠️ 3

⚠️ Lint warnings — fix before merging

The following datasources translated successfully but the static lint flagged issues that need human fixes. The auto-generated code was spliced into the JS module as-is; please patch the file directly in this PR.

datasources/xiaohongshu/search_rednote.py -> modules/rednote.js

  • [map_item_function] Even without an f prefix, "text {var}" / 'text {var}' are literal text in JavaScript — no interpolation happens. Whenever the original Python used an f-string, the JS must use a template literal (backticks).
  • [map_item_function] Literal newline inside a string literal — JS strings can't span lines without escape ("\n") or template literals (`\n`).
  • [map_item_function] Regex detected. The current LLM translates regex unreliably (escapes, character classes, flags) — please verify the regex behavior against the Python original by hand.

datasources/xiaohongshu/search_rednote.py -> modules/rednote.js (35.19s) ⚠️

LLM commentary:

Implemented all static helper methods from the Python class as plain functions: map_item_from_json_api_explore, getVideoUrl, map_item_from_json_embedded, and map_item_from_html. Used the global helpers (MappedItem, MissingMappedField, MapItemException, normalize_url_encoding, py_get, formatUtcTimestamp) as required. Regex for hashtags mirrors the Python pattern using matchAll and captures the group. Image handling follows the same fallback logic, returning a MissingMappedField when no images are found. Timestamp conversion uses formatUtcTimestamp for human‑readable strings and Math.floor for unix seconds. All .get calls are replaced with py_get, and f‑strings become template literals. No additional imports are needed.

Python diff
diff --git a/datasources/xiaohongshu/search_rednote.py b/datasources/xiaohongshu/search_rednote.py
index 27c5c930..13350a0d 100644
--- a/datasources/xiaohongshu/search_rednote.py
+++ b/datasources/xiaohongshu/search_rednote.py
@@ -107,7 +107,7 @@ class SearchRedNote(Search):
         # permalinks need this token to work, else you get a 404 not found
         xsec_bit = f"?xsec_token={post['xsec_token']}" if post.get("xsec_token") else ""
         if item.get("video", {}).get("media"):
-            video_url = item["video"]["media"]["stream"]["h264"][0]["master_url"]
+            video_url = SearchRedNote.get_video_url(item["video"]["media"].get("stream") or {})
         else:
             video_url = MissingMappedField("")
 
@@ -136,6 +136,29 @@ class SearchRedNote(Search):
             "unix_timestamp": int(timestamp / 1000) if timestamp else MissingMappedField(""),
         })
 
+    @staticmethod
+    def get_video_url(streams):
+        """
+        Get the URL of a post's video
+
+        RedNote offers each video in several versions, grouped by video
+        format. Older data names these groups after the format ("h264",
+        "h265"); newer data uses codes instead ("EF4", "EF5"). H.264 plays in
+        the most browsers and video players, so use it when it is there, and
+        otherwise the first version of any format that has a URL.
+
+        :param dict streams:  The post's `video.media.stream` object
+        :return str|MissingMappedField:
+        """
+        h264 = ("h264", "EF4")
+        formats = [*h264, *[name for name in streams if name not in h264]]
+        for name in formats:
+            for stream in streams.get(name) or []:
+                if stream.get("master_url"):
+                    return stream["master_url"]
+
+        return MissingMappedField("")
+
     @staticmethod
     def map_item_from_json_embedded(item):
         """

@dale-wahl
dale-wahl marked this pull request as ready for review October 21, 2026 15:08
@dale-wahl
dale-wahl merged commit 44c5161 into master Oct 21, 2026
1 check passed
@dale-wahl
dale-wahl deleted the auto/4cat-map-item-sync-xiaohongshu branch October 21, 2026 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant