Move support macros to attribute procedural macros

[gui1117]

This issue is about moving decl_* to one simple module with attribute macro, inspired by ink macros.
goal is readability, understandability and IDE support, I test with rust-analyzer on vscode.

the idea is to have something like

#[frame_support::pallet(Example)]
pub mod pallet {
	#[pallet::trait_]
	pub trait Trait: frame_system::Trait {
		#[pallet::const_]
		type MyGetParam: Get<u32>;
	}

	#[pallet::module]
	pub struct Module<T, I>(core::marker::PhantomData<T, I>);

.....
}

Precise design should now be discussed in the PR #6877

[gui1117]

Good point from @bkchr IDE support is quite poor for rust macro for the moment and proper handling of proc-macro expansion from IDE could come.

I conclude myself that instead of optimizing syntax for having function information for storages, maybe better to optimize readibility/understandability and remove as much boilerplate as possible.

[bkchr]

goal is readability, understandability and IDE support, I test with rust-analyzer on vscode.

IDE support should be not the goal here. If the compiler accepts it, it is valid code. This means the IDE needs to improve and I don't want that we write code that makes rust-analyzer happy. If they finished their proc-macro support and something does not work, we should open an upstream issue.

(Too slow)

[bkchr]

Regarding storage, we could switch either to something like this:

#[storage]
struct Storage {
    my_item: Map,
    my_other_item: Value,
}

Which gives you access to the stuff by doing:

Storage::my_value().get();

Storage::my_other_value().set("hey");

We would return some fake instances that still write data directly to the storage as it is done currently.

Or if we want to split up the storage items, as you proposed, I would do the following:

#[storage(map)]
struct TotalBalance<T>(T::Balance);

Regarding putting everything into a module, I'm not a real fan of it. I understand that there is currently no other way, but I don't really like it :D

[kianenigma]

Pretty old but might still be related: https://hackmd.io/n_tTUbpMQiKiG7F2dURb6g?view

[gui1117]

could relate #5678

[kianenigma]

@thiolliere you probably made a mistake here? you are referencing the same issue 😄

could relate #5678

[gui1117]

@thiolliere you probably made a mistake here? you are referencing the same issue smile

could relate #5678

aah :-/ it was this one #5040

Also I have a WIP branch https://github.com/paritytech/substrate/compare/gui-macro-attribute?expand=1

I still have to figure out how we want to do storage declararion (and try to reuse current proc_macro implementation) and add some tests.

[gui1117]

EDIT: see message below

[gui1117]

this is a new list of ideas for decl_storage, then I'll go for implementation and should be quite in good shape IMHO

EDIT: people seems to have voted mainly on 2

1 - do as of now: specific and concise syntax

each storage declared as:
decl_storage!(MyStorage get(fn foo) config(): map u32 => u32)
or multiple one at once:

decl_storage!(
  MyStorage get(fn foo) config(): map u32 => u32;
  MyStorage2 get(fn foo) config(): map u32 => u32;
)

2 - ask user to implement a rust trait helped by macro:

user implement a real rust trait but some associated type are expanded by macro

the full rust trait is:

trait StorageMap {
    /// Key type to insert
    type Key: Codec;

    /// Hasher to use
    type Hasher: StoragHasher;

    /// Query type to store
    type Query: Codec;

    /// If not provided, macro will expand to Query::default().
    ///
    /// Default returned value when storage is empty
    fn default() -> Self::Query;
    // NOTE: this was before the syntax `= ..` before

    /// Automatically filled by macro: if Query is `Option<$something>` then value is only inner
    /// type, otherwise Query == Value
    ///
    /// Value stored on chain.
    type Value: Codec;

    /// Automatically filled by macro (using function default above)
    fn from_optional_value_to_query(v: Option<Self::Value>) -> Self::Query;
    /// Automatically filled by macro.
    fn from_query_to_optional_value(v: Self::Query) -> Option<Self::Value>;

    /// Automatically filled by macro (using the name of the pallet)
    fn module_prefix() -> &'static [u8];

    /// Automatically filled by macro (using the name of the storage)
    fn storage_prefix() -> &'static [u8];

    // All operation on storage are automatically implemented in rust not by macro but at trait
    // definition as currently the case in substrate.

    fn get(..) -> .. { ..}
    fn remove(..) -> .. { ..}
    fn mutate(..) -> .. { ..}
    ...
}

user use it like this:

#[pallet::storagedef] // Expand phantom data automatically
pub struct MyStorage<T>;
#[pallet::storageimpl] // Expand some associated type in the trait automatically
impl<T: Trait> StorageMap for MyStorage<T> {
    type Key = u32;
    type Hasher = Blake2_128;
    type Query =  u32;
    fn default() -> Self::Value { 3u8 }
    .. // Here macro expand prefix and other stuff
}

or this:

#[pallet::storagedef] // Expand phantom data automatically
pub struct MyStorage<T>;
#[pallet::storageimpl] // Expand some associated type in the trait automatically
impl<T: Trait> StorageMap for MyStorage<T> {
    type Key = u32;
    type Hasher = Blake2_128;
    type Query =  Option<u32>;
    .. // Here macro expand default and other stuff
}

3 - same as 2 but with slightly less boilerplate

#[pallet::storage] // create struct, expand some associated type in the trait automatically
impl<T: Trait> StorageMap for MyStorage<T> {
    type Key = u32;
    type Hasher = Blake2_128;
    type Query =  u32;
    fn default() -> Self::Value { 3u8 }
    .. // Here macro expand prefix and other stuff
}

or this:

#[pallet::storage] // create struct, expand some associated type in the trait automatically
impl<T: Trait> StorageMap for MyStorage<T> {
    type Key = u32;
    type Hasher = Blake2_128;
    type Query =  Option<u32>;
    .. // Here macro expand default and other stuff
}

4 - ask user to implement a real rust trait with no help from macro.

user implement a real rust trait, macro use this to implement the full storage map trait.

/// value stored is Value, value queried (with get and put) is Option<Value>
trait StorageMapImplOption: StorageIdentifier {
    type Key: Codec;
    type Hasher: StorageHasher;
    type Value;
}

/// value stored is Value, value queried (with get and put) is Value (or default if None)
trait StorageMapImplUnwrapDefault: StorageIdentifier {
    type Key: Codec;
    type Hasher: StorageHasher;
    type Value;
    fn default() -> Value;
}

/// Implemented by macro using Instance and/or pallet name
trait StorageIdentifier {
    fn module_prefix () -> ...
    fn storage_prefix () -> ...
}

implemented like this:

#[pallet::storagedef] // Expand phantom data automatically, and implement StorageIdentfier.
pub struct MyStorage<T>;
#[pallet::storageimpl] // Expand metadata and implement StorageMap trait which contains method
impl<T: Trait> StorageMapImplImplUnwrapDefault for MyStorage<T> {
    type Key = u32;
    type Hasher = Blake2_128;
    type Value =  u32;
    fn default() -> Self::Value { 3u8 }
}

5 - same as 4 but no need to declare struct

user implement a real rust trait, macro use this to implement the full storage map trait.

// Expand metadata and implement StorageMap trait which contains method
// Expand storage struct and implement StorageIdentifier on it.
#[pallet::storage]
impl<T: Trait> StorageMapImplImplUnwrapDefault for MyStorage<T> {
    type Key = u32;
    type Hasher = Blake2_128;
    type Value =  u32;
    fn default() -> Self::Value { 3u8 }
}

[kianenigma]

Honestly, I am somewhat in favour of 1 from the list of ideas above because all the rest will force each storage to be declared within multiple lines, and I think that will not be desirable.

Indeed not an important factor, but something to consider. All the other proposals will make the code section related to storage at least 5x. This is also slightly bad for readability.

[bkchr]

I also think that 1 is the best option for now.

[gui1117]

thanks to @tomusdrw for giving idea, we can also do something similar to HashMap type:

user would have to write:

#[pallet::storage]
type MyStorageWithOptionQuery = StorageValue<u32, MyStorageWithOptionQueryPrefix>;

#[pallet::storage]
type MyStorageWithDefaultValue = StorageValue<u32, MyStorageWithDefaultValuePrefix, u32>;

parameter_type!(
  pub const MyDefaultGetter: u32 = 3u32;
);
#[pallet::storage]
type MyStorageWithMyDefaultValue = StorageValue<u32, MyStorageWithMyDefaultValuePrefix, u32, MyDefaultGetter>;

and frame-support would write this:

struct GetDefault<T>(core::marker::PhantomData<T>);
trait Getter<T> { // This trait is `Get` in frame-support currently
    fn get() -> T;
}
impl<T: Default> Getter<T> for GetDefault<T> {
    fn get() -> T {
        T::default()
    }
}
struct StorageValue<V, Prefix, Q=Option<V>, D=GetDefault<Q>>(core::marker::PhantomData::<(V, Q, D, Prefix)>);

impl<V, P, D:Getter<Option<V>>> StorageValue<V, P, Option<V>, D> {
    fn get() {}
    fn put() {}
}
impl<V, P, D:Getter<V>> StorageValue<V, P, V, D> {
    fn get() {}
    fn put() {}
}

Full compiling example here: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=4823a6b588378ab7e8de952e12ecae00

EDIT: I already implemented it in the WIP branch, looks very good IMO

[bkchr]

user would have to write:

#[pallet::storage]
type MyStorageWithOptionQuery = StorageValue<u32, MyStorageWithOptionQueryPrefix>;

#[pallet::storage]
type MyStorageWithDefaultValue = StorageValue<u32, MyStorageWithDefaultValuePrefix, u32>;

parameter_type!(
  pub const MyDefaultGetter: u32 = 3u32;
);
#[pallet::storage]
type MyStorageWithMyDefaultValue = StorageValue<u32, MyStorageWithMyDefaultValuePrefix, u32, MyDefaultGetter>;

and frame-support would write this:

Looks good :)