Skip to content

gen_udev_rules: add raw partition rule generator - #156

Open
Wenwen Fu (wenwfu) wants to merge 1 commit into
qualcomm-linux:mainfrom
wenwfu:udev-raw-partition-rules
Open

gen_udev_rules: add raw partition rule generator#156
Wenwen Fu (wenwfu) wants to merge 1 commit into
qualcomm-linux:mainfrom
wenwfu:udev-raw-partition-rules

Conversation

@wenwfu

Copy link
Copy Markdown

Summary

Add a gen_udev_rules subcommand that generates machine-specific udev
rules for reviewed Qualcomm raw GPT partitions.

The generator:

  • reads one or more qcom-ptool partition layouts
  • filters labels through a packaged reviewed policy
  • emits exact partition labels present in the selected layouts
  • skips blkid only for matched raw partitions
  • preserves persistent partition metadata links
  • rejects unsafe labels and invalid policy patterns
  • exits successfully without creating an output file when no approved
    labels are present

Example:

qcom-ptool gen_udev_rules \
    -i platforms/qcs6490-rb3gen2/ufs/partitions.conf \
    -o 55-qcom-raw-partitions-noblkid.rules

Motivation

This functionality was originally implemented in
qualcomm-linux/meta-qcom#2865.

Move the generic generator, reviewed policy, and udev template into
qcom-ptool so they can be reused outside the Yocto layer. The meta-qcom
change will be updated to invoke this subcommand and retain only the
machine-specific integration and packaging.

Unknown or custom partition labels continue through the normal systemd
blkid probing path.

@wenwfu

Copy link
Copy Markdown
Author

Hi Igor Opaniuk (@igoropaniuk), a friendly review ping when you have time. This PR provides the qcom-ptool side of qualcomm-linux/meta-qcom#2865. I would particularly appreciate feedback on:

  • whether qcom-ptool is the right place for this generator and policy;
  • the CLI and packaged policy format;
  • preserving normal blkid handling for unknown or custom labels.

All current CI checks are passing. The meta-qcom integration is waiting on this interface, so an initial review or direction would be very helpful.

@igoropaniuk

Copy link
Copy Markdown
Contributor

Hi Wenwen Fu (@wenwfu) .

Hi, thanks for the ping and apologies for the delay.

Is qcom-ptool the right place?
Yes, I think so. The partition layouts are the source of truth here, and the approved-label policy is really a property of those layouts, not of the image build - deriving the rules in qcom-ptool via the existing loaders dispatcher (rather than re-parsing in meta-qcom) is the right call.

CLI and policy format
Broadly fine (argparse with repeatable -i is good).

I've review changes later today and leave comments.

@wenwfu

Copy link
Copy Markdown
Author

Hi Wenwen Fu (Wenwen Fu (@wenwfu)) .

Hi, thanks for the ping and apologies for the delay.

Is qcom-ptool the right place? Yes, I think so. The partition layouts are the source of truth here, and the approved-label policy is really a property of those layouts, not of the image build - deriving the rules in qcom-ptool via the existing loaders dispatcher (rather than re-parsing in meta-qcom) is the right call.

CLI and policy format Broadly fine (argparse with repeatable -i is good).

I've review changes later today and leave comments.

Hi Igor Opaniuk (@igoropaniuk), this PR has been rebased onto the latest main and the conflict has been resolved. All 128 local tests pass. Could you please review it when you have time? Thanks!

Comment thread qcom_ptool/gen_udev_rules.py Outdated
try:
content = generate_rules(args.inputs)
if not content:
args.output.unlink(missing_ok=True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a typo'd -o pointing at an existing unrelated file (say, another rules file in /etc/udev/rules.d/) gets deleted, with exit 0 and only a "skipped" message. Writing nothing, or refusing to touch an existing file it didn't create, would be safer.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

GOTO="qcom_raw_noblkid_end"

LABEL="qcom_raw_noblkid"
ENV{UDEV_DISABLE_PERSISTENT_STORAGE_BLKID_FLAG}="1"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will work only on recent versions of systemd.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. UDEV_DISABLE_PERSISTENT_STORAGE_BLKID_FLAG was introduced in systemd v252, so this rule does require systemd >= v252.

The older alternative is UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG, but that skips the entire 60-persistent-storage.rules file, including parent metadata imports and other persistent links. Supporting that safely would require duplicating more version-specific systemd behavior in this rule.

The blkid-specific flag is narrower: it skips only filesystem probing while preserving the rest of the standard persistent-storage processing. Therefore I would prefer to keep it and document the systemd >= v252 requirement. Do you expect this generator to support pre-v252 systemd as well?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current meta-qcom integration uses OE-Core with systemd v259, so the flag is supported by its intended consumer.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about other distros? Debian trixie? Ubuntu LTS?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. The current rule relies on UDEV_DISABLE_PERSISTENT_STORAGE_BLKID_FLAG, which was added in systemd v252. Debian Trixie and Ubuntu 24.04 LTS are covered, but Ubuntu 22.04 LTS is not.

I will either add a compatibility path for older systemd versions, or document systemd v252 as the minimum requirement and make sure the integration does not silently claim support for older systems. I agree that the current version should make this requirement explicit.

Comment thread qcom_ptool/gen_udev_rules.py Outdated
TEMPLATE_FILE = DATA_DIR / "55-qcom-raw-partitions-noblkid.rules.in"
RULES_PLACEHOLDER = "@QCOM_RAW_PARTITION_RULES@"
LABEL_RE = re.compile(r"[A-Za-z0-9_.+-]+")
PATTERN_RE = re.compile(r"[A-Za-z0-9_.+*?\[\]-]+")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that PATTERN_RE rejects [!...] negation globs, which udev and fnmatch both support

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment thread qcom_ptool/data/55-qcom-raw-partitions-noblkid.rules.in
@igoropaniuk

Copy link
Copy Markdown
Contributor

LGTM

@wenwfu

Copy link
Copy Markdown
Author

Thanks Igor Opaniuk (@igoropaniuk) for the review! All comments have been addressed. Dmitry Baryshkov (@lumag), could you please take a final look and approve/merge when convenient?

# SPDX-License-Identifier: BSD-3-Clause

# Partition-name patterns reviewed as safe to exclude from blkid probing.
# Keep one udev glob per line. A new partition label is not implicitly safe.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't like having this list of partitions. I though that the idea was to generate it on the fly. If so, it should not be a part of the commit.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated rules are still machine-specific: the generator collects the partition labels from the selected layouts and emits rules only for labels present in those layouts.

approved-raw-partition-patterns.list is not generated output; it is the reviewed allowlist used to decide which labels are safe to exclude from blkid. We cannot generate the rules from all layout labels blindly, since that could accidentally disable probing for filesystem or custom partitions.

If you prefer this policy to be owned by meta-qcom rather than packaged in qcom-ptool, I can make it an explicit policy input to the generator. Do you have a preference?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't generate the rules per the machine. A single rootfs needs to support multiple machine configurations.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't generate the rules per the machine. A single rootfs needs to support multiple machine configurations.

Thanks, I understand the concern.
In the current meta-qcom integration, the rules are intentionally machine-specific:

  • qcom-raw-partitions-udev-rules is packaged as MACHINE_ARCH.
  • The generator receives the partition layouts selected by the current MACHINE through QCOM_PARTITION_FILES_SUBDIR and QCOM_PARTITION_FILES_SUBDIR_SPINOR.
  • The generated rule is installed only in that machine's image.

Therefore, we are not currently treating one rootfs artifact as reusable across multiple MACHINEs.
If the intended requirement is that a generic rootfs must be reusable across multiple MACHINEs, I agree that the rules should be generated from the union of all supported layouts, or from a common approved policy.
Could you confirm that this is the expected image model for meta-qcom?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The expected model is that the rootfs should be as unified as possible. Other QLI distros (Debian) don't have machine-specific packages, so the generated rules, if they are to be usable, should be generic.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will break for the USB or uSD devices attached to the system. However you can use that filter to generate a set of rules from the existing conf files. As we discussed beforehand, skip the occasional one-off cases.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I understand the proposal: use the small filesystem-label filter only at generation time, scan the existing upstream partition conf files, skip the occasional one-off cases, and emit a generic set of explicit PARTNAME rules. This avoids a runtime catch-all that would affect USB or uSD devices.

One concern remains: if a downstream/custom partitions.conf adds a filesystem partition under a new name, generating by subtraction will classify it as raw and disable blkid. This is the unknown/custom-label case I mentioned earlier.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me restate the proposed approach to make sure I understand it correctly:

  1. Scan all supported in-tree partitions.conf files, rather than layouts selected for one machine.
  2. Exclude a small fixed set of filesystem-capable partition names.
  3. Ignore legacy or occasional one-off layouts/labels.
  4. Deduplicate the remaining labels and emit a generic set of explicit PARTNAME rules.
  5. Do not use an inverted catch-all rule at udev runtime.

Is this the intended design?

Two details are still unclear:

  • Which layouts or labels should be considered one-offs and excluded?
  • If a downstream/custom layout adds a filesystem partition with a new name, should it be classified as raw, or should it retain normal blkid probing?

I would like to clarify these points before reworking the generator again, since the latter choice determines whether unknown/custom layouts remain fail-safe.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You would already have a list of exclusions, like rootfs or esp. If necessary, that list can be extended.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, updated. The generator now scans all in-tree partition layouts and uses a small filesystem exclusion list, including rootfs and efi. All other discovered partitions get explicit raw-partition rules.

@wenwfu

Copy link
Copy Markdown
Author

Hi Dmitry Baryshkov (@lumag), updated. The generator now produces a single generic rule set without machine-specific layout inputs. All tests pass and the generated rules pass udevadm verify. Could you please take another look?

@wenwfu

Copy link
Copy Markdown
Author

Hi Dmitry Baryshkov (@lumag), I updated the PR based on your feedback.

The generator now scans all in-tree platforms/*/*/partitions.conf layouts by default, derives the union of partition names, and maintains only a small filesystem-name list. Unknown partition names retain normal blkid probing.

All 130 tests pass, and the generated rules pass udevadm verify. Could you please take another look and let me know whether this matches the intended design?

Generate udev rules from the union of all in-tree partition layouts by
default, while allowing repeatable -i options for selected layouts.

Keep a small reviewed list of filesystem partition names and emit exact
PARTNAME matches for the remaining labels. Names outside the supplied
layouts are not emitted, so unknown downstream partitions retain normal
blkid probing.

This keeps the generated rules machine-independent and avoids a runtime
catch-all rule.

Signed-off-by: Wenwen Fu <wenwfu@qti.qualcomm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants