Allow inline configuration vs file paths

Docs/Examples/DebugDump/config.yml

@@ -12,18 +12,27 @@
 # THIS WILL GENERATE A LOT OF DATA, so use it with care.
 #
 # If you want to enable filtering and/or PII data, uncomment the
-# correpsonding lines and create the additional .yml files.
+# corresponding lines below.
+#
+# You can also use ${file:PATH} to reference an external YAML file,
+# e.g.: filter: "${file:/path/to/filter.yml}"
+#
+# For backwards compatibility, you can also specify a plain file path:
+# e.g.: filter: "/path/to/filter.yml"
 
 receivers:
   trace2receiver:
     socket: "/usr/local/<my-install-dir>/trace2.socket"
     pipe:   "//./pipe/<my-pipe-name>"
 
-#   filter: "/usr/local/<my-install-dir>/filter.yml"
-#   pii:    "/usr/local/<my-install-dir>/pii.yml"
+#   pii:
+#     include:
+#       hostname: true
+#       username: false
 
-#   filter: "C:/ProgramData/<my-data-dir>/filter.yml"
-#   pii:    "C:/ProgramData/<my-data-dir>/pii.yml"
+#   filter:
+#     defaults:
+#       ruleset: "dl:verbose"
 
 processors:
 

Docs/Examples/ExportToAzureMonitor/config.yml

@@ -11,18 +11,27 @@
 # THIS WILL GENERATE A LOT OF DATA, so use it with care.
 #
 # If you want to enable filtering and/or PII data, uncomment the
-# correpsonding lines and create the additional .yml files.
+# corresponding lines below.
+#
+# You can also use ${file:PATH} to reference an external YAML file,
+# e.g.: filter: "${file:/path/to/filter.yml}"
+#
+# For backwards compatibility, you can also specify a plain file path:
+# e.g.: filter: "/path/to/filter.yml"
 
 receivers:
   trace2receiver:
     socket: "/usr/local/<my-install-dir>/trace2.socket"
     pipe:   "//./pipe/<my-pipe-name>"
 
-#   filter: "/usr/local/<my-install-dir>/filter.yml"
-#   pii:    "/usr/local/<my-install-dir>/pii.yml"
+#   pii:
+#     include:
+#       hostname: true
+#       username: false
 
-#   filter: "C:/ProgramData/<my-data-dir>/filter.yml"
-#   pii:    "C:/ProgramData/<my-data-dir>/pii.yml"
+#   filter:
+#     defaults:
+#       ruleset: "dl:verbose"
 
 processors:
 

Docs/config-filter-settings.md

@@ -1,14 +1,20 @@
 # Config Filter Settings
 
-The `filter.yml` file controls how the `trace2receiver` component
+The filter settings control how the `trace2receiver` component
 translates the Trace2 data stream from Git commands into OTEL data
 structures.  This filtering is content- and context-aware and is
 independent of any statistical filtering performed by later stages in
 the OTEL Collector pipeline.
 
-The filter settings pathname is set in the
+The filter settings are specified inline under the
 `receivers.trace2receiver.filter`
-parameter in the main `config.yml` file.
+parameter in the main `config.yml` file.  Alternatively, you can use
+the `${file:PATH}` syntax to reference an external YAML file.
+
+For backwards compatibility, you can also specify a plain file path
+string (without the `${file:}` wrapper) as the value of the `filter`
+field, and the receiver will read and parse the YAML file at that
+path.
 
 
 
@@ -90,14 +96,15 @@ A ruleset name is essentially an alias for the underlying ruleset
 file.  Using a ruleset name avoids requiring users know how and where
 the telemetry service is installed.
 
-The `filter.yml` file contains a dictionary to map ruleset names to
+The filter settings contain a dictionary to map ruleset names to
 pathnames:
 
 ```
-rulesets:
-  <ruleset-name-1>: <ruleset-pathname-1>
-  <ruleset-name-2>: <ruleset-pathname-2>
-  ...
+filter:
+  rulesets:
+    <ruleset-name-1>: <ruleset-pathname-1>
+    <ruleset-name-2>: <ruleset-pathname-2>
+    ...
 ```
 
 Ruleset files will be loaded when the receiver starts up.
@@ -127,14 +134,15 @@ the ruleset "rs:bar".
 
 A repo nickname is a simple string without either `dl:` or `rs:` prefix.
 
-The `filter.yml` file contains a dictionary to map nicknames to detail
+The filter settings contain a dictionary to map nicknames to detail
 levels or rulesets:
 
 ```
-nicknames:
-  <nickname-1>: <ruleset-name> | <detail-level>
-  <nickname-1>: <ruleset-name> | <detail-level>
-  ...
+filter:
+  nicknames:
+    <nickname-1>: <ruleset-name> | <detail-level>
+    <nickname-1>: <ruleset-name> | <detail-level>
+    ...
 ```
 
 
@@ -160,13 +168,14 @@ or `system` level.
 $ git config --system trace2.configparams "otel.trace2.*"
 ```
 
-The `filter.yml` contains a dictionary to define the spelling of
+The filter settings contain a dictionary to define the spelling of
 these keys:
 
 ```
-keynames:
-  nickname_key: "otel.trace2.nickname"
-  ruleset_key:  "otel.trace2.ruleset"
+filter:
+  keynames:
+    nickname_key: "otel.trace2.nickname"
+    ruleset_key:  "otel.trace2.ruleset"
 ```
 
 
@@ -199,7 +208,7 @@ $ git -c otel.trace2.nickname=personal status
 ```
 
 If no nickname is defined or the given repo nickname is not defined in
-the `filter.yml` file, the receiver will fall back to the default
+the filter settings, the receiver will fall back to the default
 filter settings.
 
 _In the above example, I've suggested "monorepo" and "personal" as
@@ -244,8 +253,8 @@ $ cd /path/to/my/repo4
 $ git -c otel.trace2.ruleset="dl:summary" status
 ```
 
-If the named ruleset or detail level is not defined in the `filter.yml`
-file, the receiver will fall back to the default filter settings.
+If the named ruleset or detail level is not defined in the filter
+settings, the receiver will fall back to the default filter settings.
 
 If a Git command sends both a `ruleset_key` and `nickname_key`, the
 `ruleset_key` wins.  (Both key values will be included in the OTEL
@@ -298,9 +307,42 @@ This would produce the following in the OTEL process span:
 ## Filter Settings Syntax
 
 Now that all of the concepts have been introduced, we can describe
-the complete syntax of the `filter.yml` file.  All sections and rows
+the complete syntax of the filter settings.  All sections and rows
 are optional.
 
+When the settings are specified inline in the Collector configuration,
+they appear under the receiver's `filter` field:
+
+```
+filter:
+  keynames:
+    nickname_key: <git-config-key>
+    ruleset_key:  <git-config-key>
+
+  nicknames:
+    <nickname-1>: <ruleset-name> | <detail-level>
+    <nickname-1>: <ruleset-name> | <detail-level>
+    ...
+
+  rulesets:
+    <ruleset-name-1>: <ruleset-pathname-1>
+    <ruleset-name-2>: <ruleset-pathname-2>
+    ...
+
+  defaults:
+    ruleset: <ruleset-name> | <detail-level>
+
+  important_events:
+    - category: <category-string>
+      key_prefix: <key-prefix-string>
+      field_name: <summary-field-name>
+    ...
+```
+
+When the `filter` field references a standalone file using either a
+plain path or `${file:PATH}`, the file contains the settings directly
+and omits the outer `filter` field:
+
 ```
 keynames:
   nickname_key: <git-config-key>
@@ -339,19 +381,20 @@ used.
 In this filter:
 
 ```
-keynames:
-  nickname_key: "otel.trace2.nickname"
-  ruleset_key:  "otel.trace2.ruleset"
+filter:
+  keynames:
+    nickname_key: "otel.trace2.nickname"
+    ruleset_key:  "otel.trace2.ruleset"
 
-nicknames:
-  monorepo: "dl:verbose"
-  personal: "dl:drop"
+  nicknames:
+    monorepo: "dl:verbose"
+    personal: "dl:drop"
 
-rulesets:
-  "rs:status": "./rulesets/rs-status.yml"
+  rulesets:
+    "rs:status": "./rulesets/rs-status.yml"
 
-defaults:
-  ruleset: "dl:summary"
+  defaults:
+    ruleset: "dl:summary"
 ```
 
 The receiver will watch for the `otel.trace2.nickname` and
@@ -371,4 +414,3 @@ use `dl:drop` and not emit any telemetry.
 All other commands will use the default `dl:summary` and
 emit command overview telemetry.
 
-

Docs/config-pii-settings.md

@@ -1,6 +1,6 @@
 # Config PII Settings
 
-The PII Settings file contains privacy-related feature flags for the
+The PII settings contain privacy-related feature flags for the
 `trace2receiver` component.  Currently, this includes flags to add
 user and hostname data that may not be present in the original Trace2
 data stream.  Later, it may include other flags to redact or not
@@ -9,13 +9,31 @@ redact sensitive data found within the Trace2 data stream.
 NOTE: These flags may add GDPR-sensitive data to the OTEL telemetry
 data stream.  Use them at your own risk.
 
-The PII settings pathname is set in the
+The PII settings are specified inline under the
 `receivers.trace2receiver.pii`
-parameter in the main `config.yml` file.
+parameter in the main `config.yml` file.  Alternatively, you can use
+the `${file:PATH}` syntax to reference an external YAML file.
 
-## `pii.yml` Syntax
+For backwards compatibility, you can also specify a plain file path
+string (without the `${file:}` wrapper) as the value of the `pii`
+field, and the receiver will read and parse the YAML file at that
+path.
 
-The PII settings file has the following syntax:
+## PII Settings Syntax
+
+When the settings are specified inline in the Collector configuration,
+they appear under the receiver's `pii` field:
+
+```
+pii:
+  include:
+    hostname: <bool>
+    username: <bool>
+```
+
+When the `pii` field references a standalone file using either a plain
+path or `${file:PATH}`, the file contains the settings directly and
+omits the outer `pii` field:
 
 ```
 include: