Add rule hint header

[joaotgouveia]

Previously, cpp-rule-preprocessor worked by synthesizing the template arguments required by a given template rule and forcing its instantiation, sidestepping type checking. In the presence of hints, the synthesized types would inherit the hints, which also introduced problems. This solution was neither robust nor easily extensible.

This PR adds a header containing hints that can be used by rule authors and refactors how the preprocessor works. Each template rule is now directly instantiated using the appropriate Sema APIs. Additionally, compilation errors are no longer suppressed, a compilation error that arises during preprocessing now means that a rule is malformed. This makes the preprocessor much simpler and more robust.

The template arguments used to instantiate each rule are created based on the provided hints. If a template parameter does not have a default, a POD is synthesized. If a parameter has a default, that default is wrapped in a type alias and used directly.

The following rule:

template <typename T1, typename T2 = ImplicitlyConvertible>
std::vector<T1> f37(T2 *first, T2 *last) {
  return std::vector<T1>(first, last);
}

Is instantiated by the preprocessor by synthesizing the following:

namespace {
struct T1 {};
using T2 = ImplicitlyConvertible;
template <T1, T2> std::vector<T1> f37(T2 *first, T2 *last);
};

These changes to the preprocessor mean that, going forward, any rule that cannot be instantiated using a POD must use an appropriate hint.

Hints that correspond to classes are printed in their generic form, T<digit>, by attaching a PreferredName attribute to the declaration. This does not work for hints defined as builtin types. These hints are instead printed as T<digit> by having normalizeQualType search for a corresponding typedef in the enclosing namespace. This lookup is only performed for SubstTemplateTypeParmType types, which ensures that there are no collisions between builtin hints and actual builtin types spelled in the rule. This is tested by the std::byte rules this patch adds.

Hints that expose inner type aliases, such as Iterator, must be parameterized over those types. Otherwise, these types are leaked into the IR. For instance, a rule such as the following:

template <typename T2, typename T1 = Iterator<T2, Long>>
typename std::reverse_iterator<T1>::reference
f4(const std::reverse_iterator<T1> &a0) {
  return a0.operator*();
}

Is currently represented as T2 & std::reverse_iterator<T1>::operator*() const in the IR. If we were to use a fixed type for the Iterator hint's reference type alias, such as using reference = long &, this rule would be incorrectly printed as long & std::reverse_iterator<T1>::operator*() const. This is tested by the std::reverse_iterator rules added by this patch. If a given rule does not rely on an inner type alias, the extra template parameter can be omitted:

template <typename T1 = Iterator<Comparable, Long>> void f1(T1 first, T1 last) {
  return std::sort(first, last);
}

The __COUNTER__ macro is used to ensure that every use of a given hint in a rule acts as a unique type, reflecting the intended semantics when declaring multiple template parameters.

I'm working on a utility that automatically generates these src.cpp files. Hints are defined using a set of macros that ensure each hint is properly annotated with the information required by this synthesizer. The structs and typedefs inside the Synthesis namespace are also used by this utility. These constructs were included in the file to avoid having to duplicate rule_hints.h in my project. All remaining synthesizer-specific helpers are kept separate.

Additionally, this patch was tested by confirming that the pre-existing IR remained unchanged. These changes to the preprocessor allow it to preprocess 1911 automatically generated rules.

Other changes:

• Cleaned up the matching logic in the preprocessor, which was essentially a duplicate of the logic in Mapper::ToString.
• Changed how the Rust rule preprocessor handles where clauses. Previously, the preprocessor iterated over each predicate in the clause and added any types in it to the list of generics. This meant that, for clauses such as where u8: std::ops::Shl<T1, Output = u8>, u8 was incorrectly added as a generic parameter. This patch changes this behavior, types in the where clause that are not present on the generics list are now skipped. I believe this is correct, since a where clause cannot be used to declare new generics anyway.
• Added the UnsafeIterator trait, which is useful for writing rules where we know that a given generic is an iterator that will be translated to Ptr.
• Changed how operators that are spelled using < or > are printed. Printing < or > collided with the search performed by Mapper::matchTemplate.

[lucic71]

This PR is too big. I think it's better to split it in:

1. operator<< -> shl fix in Mapper
2. The where change in rule-preprocessor. I find the where u8: Shl<T1, Output = u8> spelling weird. I thought the preprocessor only accepts generics that are named T1, T2, etc
3. UnsafeIterator should be something like PtrIterator that is implemented for *mut T/*const T/Ptr<T>. Also we use iterators in other places, like std::vector::iterator, those should probably pe translated as PtrIterator as well. Every random access iterator can be a PtrIterator. Similar to how stable iterators are currently translated as MapIterator
4. The hints infrastructure, basically the remaining part of the PR

I have some comments about 4.:

• I see that you declared some hints that are not currently used. If I got my numbers right, only 40% of the hints are usde in the rules and in rule_hints.h. I think it's better (for reviewing) to upstream only the used ones for the moment, then add the rest when they are actually needed
• I see that you use the __COUNTER__ + createAliasType to have unique typenames per rule. I also see that for builtin types this doesn't work so you pass DeclContext to ToString in order to search the type alias in the name space. However I think the strategy for builtins is not fully correct:
  • template <typename T1 = Integer, typename T2 = Integer> bool f1(T1 a, T2 b) { ... }. The preprocessor emits using T1 = int; using T2 = int, normalizeQualType, for T2, does find_if on {T1, T2} for an int and finds T1 instead of T2. This will result in bool f1(T1, T1) meaning that bool f1(int, long) will never be matched
  • template <typename T1 = Integer> std::pair<T1, T1> f2(T1 a) { ... }. The return type is TemplateSpecializationType{pair, Subst(T1 -> int), Subst(T2 -> int)} and the isa<SubstTemplateTypeParmType> fails meaning that the rule is printed as std::pair<int, int> f2(int a) { ... } instead of std::pair<T1, T1> ...

I was thinking that maybe you can use a mapping similar to {T1 -> _BitInt(1), T2 -> _BitInt(2), ...} or {T1 -> enum T1: int {}, T2 -> enum T2: int {}, ...}, but they don't satisfy the std::is_integral trait. So those 2 solutions + the solution that you have in this PR are not good

[lucic71]

Also, after #302 is merged, you should modify the docs about cpp-rule-preprocessor and rules

[joaotgouveia]

I'll close this and split it into smaller PRs then