Align with JSON-RPC?

[tlrobinson]

I think Matt mentioned this early on: what we have is fairly close to JSON-RPC. Maybe it's worth making it compatible?

JSON-RPC 1.0 uses an array of arguments, but 2.0 supports named parameters, so it would be almost identical to what we have (rename inputs to params, outputs to result).

One nice thing about using JSON-RPC is we could potentially expose WebPipes over more efficient transports like TCP sockets/WebSockets/ZeroMQ. When using HTTP we could drop the id and method field (since the "method" is in the URL). When using a socket, the method would just be the url or path, and id would be used to pair the response with the request.

[progrium]

Not a bad idea.

On Sun, Dec 2, 2012 at 1:19 AM, Tom Robinson notifications@github.comwrote:

I think Matt mentioned this early on: what we have is fairly close to
JSON-RPC. Maybe it's worth making it compatible?

JSON-RPC 1.0 uses an array of arguments, but 2.0 supports named
parameters, so it would be almost identical to what we have (rename inputsto
params, outputs to result).

One nice thing about using JSON-RPC is we could potentially expose
WebPipes over more efficient transports like TCP sockets/WebSockets/ZeroMQ.
When using HTTP we could drop the id and method field (since the "method"
is in the URL). When using a socket, the method would just be the url or
path, and id would be used to pair the response with the request.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4.

Jeff Lindsay
http://progrium.com

[matthewhudson]

I approve. It's also a bit easier to explain to developers as "Federated RPC".

The id field is nice since it essentially behaves as a token/key which is useful in cases where a Block needs to handle state.

[tlrobinson]

I don't think the JSON-RPC id field is useful for tracking state since it's
unique per request. I think we'll need another way to identify the pipeline.

On Dec 2, 2012, at 9:42 AM, Matthew Hudson notifications@github.com wrote:

I approve. It's also a bit easier to explain to developers as "Federated
RPC".

The id field is nice since it essentially behaves as a token/key which is
useful in cases where a Block needs to handle state.

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/4#issuecomment-10932422.

[progrium]

Headers. Either HTTP or if new JSON-RPC supports some kind of headers.

On Sun, Dec 2, 2012 at 9:45 AM, Tom Robinson notifications@github.comwrote:

I don't think the JSON-RPC id field is useful for tracking state since
it's
unique per request. I think we'll need another way to identify the
pipeline.

On Dec 2, 2012, at 9:42 AM, Matthew Hudson notifications@github.com
wrote:

I approve. It's also a bit easier to explain to developers as "Federated
RPC".

The id field is nice since it essentially behaves as a token/key which is
useful in cases where a Block needs to handle state.

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/4#issuecomment-10932422.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-10932448.

Jeff Lindsay
http://progrium.com

[matthewhudson]

A database Block perfectly illustrates the value of a "methods" key in the Block Definition.

{
    "name": "database",
    "description": "Store and fetch persistent data.",
    "url": "/block-endpoint",
    "methods": {
        "get": {
            "inputs": [],
            "outputs": []
        },
        "set": {
            "inputs": [],
            "outputs": []
        }
    }
}

To accomodate this new Block Definition format, a HTTP request would change in the following ways:

• The "inputs" wrapper would take the name of the method invoked.
• As of WebPipe Spec v0.2 we no longer use the 'inputs' wrapper array.

curl -i -X POST -d '{"methodName":{"inputName":"inputValue", ... }}' -H "Content-Type: application/json" http://block-endpoint

Thoughts?

[matthewhudson]

Or, if we went full JSON-RPC:

curl -i -X POST -d '{ "method": "get", "params": ["database_key"], "id": null }' -H "Content-Type: application/json" http://block-endpoint

Using the database Block Definition I mentioned in my last post, this example would return a record for a given database key.

[tlrobinson]

Including a method name in the request body is very un-RESTful, but then again what we have so far isn't exactly RESTful either (and JSON-RPC definitely isn't either).

It would be nice if we could just use GET methods to enable caching, etc, but GET requests don't have a request body for us to include the inputs. Inputs could be mapped to the query string (or even include the JSON in the URL).

The HTTP methods could be mapped to the method field for other JSON-RPC transports. If more than one block were handled by a single JSON-RPC server the method could include the url. GET:/my-database or something.

Also we definitely shouldn't use JSON-RPC 1.0 positional arguments. JSON-RPC 2.0 supports named arguments.

[matthewhudson]

I'm sorry, I wasn't as clear as I could have been with my database example. We aren't passing a HTTP Method as an argument, we're passing the name of the "remote procedure" we want to run. In JSON-RPC, the "method" is the remote procedure you want to invoke.

In my example, the database block exposed two methods/functions/remote procedures - 'get' and 'set'. 'Get' would return a database record, and 'set' would save a database record.

As for REST, we're pretty much eschewing HTTP all together if we decide to go multi-protocol. That makes the following things irrelevant:

1. HTTP Subscriptions
2. CORS
3. HTTP Methods (OPTIONS, POST)
4. HTTP Response Body
5. HTTP Status Codes signaling success/failure

[progrium]

I feel like maybe aligning with JSON-RPC is getting a little out of hand.
We did agree these are WebPipes and so it makes sense to focus on web
technologies. Using other protocols is going to have very little benefit to
the goals of the project.

On Sun, Dec 9, 2012 at 8:05 PM, Matthew Hudson notifications@github.comwrote:

I'm sorry, I wasn't as clear as I could have been with my database
example. We aren't passing a HTTP Method as an argument, we're passing the
name of the "remote procedure" we want to run. In JSON-RPC, the "method" is
the remote procedure you want to invoke.

In my example, the database block exposed two methods/functions/remote
procedures - 'get' and 'set'. 'Get' would return a database record, and
'set' would save a database record.

As for REST, we're pretty much eschewing HTTP all together if we decide to
go multi-protocol. That makes the following things irrelevant:

1. HTTP Subscriptions

2. CORS

3. HTTP Methods (OPTIONS, POST)

4. HTTP Response Body

5. HTTP Status Codes signaling success/failure

   —
   Reply to this email directly or view it on GitHubhttps://github.com/Align with JSON-RPC? #4#issuecomment-11181297.

Jeff Lindsay
http://progrium.com

[matthewhudson]

I have to agree with Jeff. I think we ought to leverage JSON-RPC only for inspiration, not total adoption. Like Jeff said, we agreed to focus on HTTP. I think the two biggest inspirations we gain from JSON-RPC are:

1. The 'error' key in output. In addition to HTTP Status Codes, this allows Blocks to provide a precise explanation for failure.
2. The 'method' key in the input. This affords Blocks the ability to expose multiple functions which is crucial for something like a database block.

[progrium]

Why can't a database block be split into different blocks?

On Sun, Dec 9, 2012 at 8:34 PM, Matthew Hudson notifications@github.comwrote:

I have to agree with Jeff. I think we ought to leverage JSON-RPC only for
inspiration, not total adoption. Like Jeff said, we agreed to focus on
HTTP. I think the two biggest inspirations we gain from JSON-RPC are:

1. The 'error' key in output. In addition to HTTP Status Codes, this
   allows Blocks to provide a precise explanation for failure.

2. The 'method' key in the input. This affords Blocks the ability to
   expose multiple functions which is crucial for something like a database
   block.

   —
   Reply to this email directly or view it on GitHubhttps://github.com/Align with JSON-RPC? #4#issuecomment-11181698.

Jeff Lindsay
http://progrium.com

[matthewhudson]

Wouldn't that be confusing? How would a block specify a relationship with one or more other blocks?

[progrium]

I don't know what you mean. Give me an example.

On Sun, Dec 9, 2012 at 8:56 PM, Matthew Hudson notifications@github.comwrote:

Wouldn't that be confusing? How would a block specify a relationship with
one or more other blocks?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-11181979.

Jeff Lindsay
http://progrium.com

[matthewhudson]

So, I'm proposing we adopt the "methods" key from JSON-RPC so that a single block can expose multiple functions. This new "methods" key would map method names to our inputs/outputs.

Here's an example:

{
    "name": "database",
    "description": "Store and fetch persistent data.",
    "url": "/block-endpoint",
    "methods": {
        "get": {
            "inputs": [],
            "outputs": []
        },
        "set": {
            "inputs": [],
            "outputs": []
        }
    }
}

You seem to be proposing that we don't need to allow blocks to expose multiple methods and that my example database block should instead be two separate blocks, like this:

Save Record:

{
    "name": "database-set-record",
    "description": "Save persistent data.",
    "url": "/block-endpoint",
    "inputs": [],
    "outputs": []

}

Retrieve Record:

{
    "name": "database-get-record",
    "description": "Fetch persistent data.",
    "url": "/block-endpoint",
    "inputs": [],
    "outputs": []
}

So my question to you is this: how does "database-set-record" describe its relationship with "database-get-record" and vice-versa? How does an enduser who knows about one of these block know about the other?

[progrium]

Naming convention? Meta-data? Maybe that's a registry/index problem?

There was an operating system called Genera. It was 100% LISP. Think about
the complexity of an operating system. Now think about there being a single
namespace for every operation. The operating system was amazing in that
every function was available to the user to explore and everything in it
was inspectable. It was truly an open operating system. If you found a get
function and wondered if there was a set function, you'd just look it up.
There were indexes and things to help you find functions. But you wouldn't
find one function from another function.

In fact, think of Unix today. There are lots of system calls with related
functions. It's in the man pages that you find the related functions.

Anyway, I'm getting too far from the point. I think the point is that the
reason you'd want a block to serve multiple functions is understandable,
but it's not necessary. The big reason why I'm against it is that every
graphical programming environment has maintained a simple relationship
between their equivalent of a block and a function. And I think it's for a
good reason.

What do you call a block with multiple functions? An aggregate block? Wait
wait, we already have that concept. So let's introduce another name that's
inevitably going to be similar? This is something I'd only want to do as a
last resort.

-jeff

On Sun, Dec 9, 2012 at 9:09 PM, Matthew Hudson notifications@github.comwrote:

So, I'm proposing we adopt the "methods" key from JSON-RPC so that a
single block can expose multiple functions. This new "methods" key would
map method names to our inputs/outputs.

Here's an example:

{
"name": "database",
"description": "Store and fetch persistent data.",
"url": "/block-endpoint",
"methods": {
"get": {
"inputs": [],
"outputs": []
},
"set": {
"inputs": [],
"outputs": []
}
}
}

You seem to be proposing that we don't need to allow blocks to expose
multiple methods and that my example database block should instead be two
separate blocks, like this:

Save Record:

{
"name": "database-set-record",
"description": "Save persistent data.",
"url": "/block-endpoint",
"inputs": [],
"outputs": []

}

Retrieve Record:

{
"name": "database-get-record",
"description": "Fetch persistent data.",
"url": "/block-endpoint",
"inputs": [],
"outputs": []
}

So my question to you is this: how does "database-set-record" describe its
relationship with "database-get-record" and vice-versa? How does an enduser
who knows about one of these block know about the other?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-11182159.

Jeff Lindsay
http://progrium.com