fix: remove duplicated standalone-year acceptor definitions

[andrewwhitecdw]

This PR addresses the following issue in nemo_text_processing/text_normalization/ko/taggers/date.py: remove duplicated standalone-year acceptor definitions.

Changes

• nemo_text_processing/text_normalization/ko/taggers/date.py: remove duplicated standalone-year acceptor definitions.

Details

--- a/nemo_text_processing/text_normalization/ko/taggers/date.py
+++ b/nemo_text_processing/text_normalization/ko/taggers/date.py
@@ -1,5 +1,2 @@
-        # For standalone years:
-        # - No era: 1–4 digits with NO leading zeros
-        YEAR_NO_ERA_1TO4 = pynini.closure(pynutil.delete("0"), 0, 3) + _1to9 + pynini.closure(_d, 0, 3)
-        # - With era (기원전/기원후): allow leading zeros but strip them
-        YEAR_ERA_1TO4 = pynini.closure(pynutil.delete("0"), 0, 3) + _1to9 + pynini.closure(_d, 0, 3)
+        # Standalone year acceptor: 1-4 digits with leading zeros stripped.
+        YEAR_1TO4 = pynini.closure(pynutil.delete("0"), 0, 3) + _1to9 + pynini.closure(_d, 0, 3)

Tests

• tests/nemo_text_processing/ko/test_date.py

--- a/tests/nemo_text_processing/ko/test_date.py
+++ b/tests/nemo_text_processing/ko/test_date.py
@@ -XX,XX +XX,XX @@ class TestDate:
         """
         self._run_test(input_text, expected)
 
+    @pytest.mark.run(after="test_date")
+    def test_standalone_year_no_era(self):
+        input_text = "2024년"
+        expected = 'date { year: "이천이십사년" }'
+        self._run_test(input_text, expected)
+
+    @pytest.mark.run(after="test_date")
+    def test_standalone_year_with_era(self):
+        input_text = "기원전233년"
+        expected = 'date { era: "기원전" year: "이백삼십삼년" }'
+        self._run_test(input_text, expected)
+
+    @pytest.mark.run(after="test_date")
+    def test_standalone_year_leading_zeros_stripped(self):
+        input_text = "기원후0024년"
+        expected = 'date { era: "기원후" year: "이십사년" }'
+        self._run_test(input_text, expected)
+
     def _run_test(self, input_text, expected):
         print("input_text: ", input_text)
         pred = self.tagger(input_text)

[github-actions]

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.