Craft story for transactions regarding REST and Spring Data REST [DATAREST-562]

Greg Turnquist opened DATAREST-562 and commented

Insert both a parent and child record in one operation/transaction.

1. Parent record is a financial expenditure, say $250 total budget.
2. Child records are line items, like $100 for training, $50 for train fare, and $100 for meals.

Kind of like:

POST /budgets
{
"name": "SpringOne 2015 trip",
"budget": "250",
"line_items": [ {
"description": "training",
"cost" : "100"
}, {
"description": "train fair",
"cost" : "50"
}, {
"description": "meals",
"cost" : "100"
}
]
}

Have this all load up two tables that are linked by a 1:N relationship, and consistent at all times.

This can be derived from the following conversation:

Jacques Chester [8:13 AM]
Good morning from NYC everyone. @olivergierke, we're available for a brief chat at your convenience.

Greg Turnquist [9:09 AM]
I believe @olivergierke is at a conference right now in Desden.

Greg Turnquist [9:10 AM]
You may need to arrange a time via email.

Jacques Chester [9:26 AM]
If he pops in, we'll take the time then. If anyone else wants to venture an opinion, that'd be great too, but it looks like Oliver was the main implementer on the relevant code.

Jacques Chester [9:26 AM]
If we can get an answer today, we'd be grateful, but we understand that everyone's needs are always the urgentest :simple_smile:

Greg Turnquist [9:51 AM]
In essence, you are indicating that your requirement is to both POST (create) two entities, and PUT one to the other, all in one RESTful operation. That doesn't sound possible. Why can't you do it the other way around? You create the subresource first, and then create the main resource with the subresource's URI in one POST?

Jacques Chester [9:55 AM]
The requirement is to create or update a subresource

Jacques Chester [9:55 AM]
we don't expect to create the toplevel resource and subresource in a single step

Jacques Chester [9:56 AM]
but we do want to create the subresource directly on the toplevel resource

Jacques Chester [9:56 AM]
the subresource is meaningless without belonging immediately to the toplevel

Greg Turnquist [9:57 AM]
If you dig into a repository solution, you'l find there are two operations happening. One to create the main entity, and then another to create the related entity. In classic DB solutions, this is wrapped in a transaction to maintain consistency. With REST, creating two resources and linking them together in another operation can be referred to as eventual consistency.

Jacques Chester [9:58 AM]
OK, but I have two problems with that argument.

Jacques Chester [9:59 AM]
The first is: eventual consistency is not acceptable for a financial application

Jacques Chester [9:59 AM]
this update is atomic

Jacques Chester [9:59 AM]
I don't want to maintain state on the REST server, because that's wasteful and not RESTful

Jacques Chester [9:59 AM]
but to do it as two HTTP operations would require me to do so to maintain the atomicity

Jacques Chester [10:00 AM]
The second is: I don't see why this is an unclear operation

Jacques Chester [10:00 AM]
if the GET on a subresource collection has clear semantics, then POST and PUT would also do so IMO

Jacques Chester [10:00 AM]
the principle of least surprise would be that it's symmetric in the same way as GET/PUT/POST are on toplevel resources

Greg Turnquist [10:02 AM]
ATMs by definition operate with eventual consistency and instead focus on availabilty. When an ATM can't talk to home, it will still give you money and catch up with a batch operation. It defrays this risk by charging you a fee if you overdraw and it couldn't guarantee the balance.

Jacques Chester [10:02 AM]
ATMs are consistent within the ATM

Jacques Chester [10:03 AM]
you're mixing up eventual consistency with partial consistency

Jacques Chester [10:03 AM]
an ATM's view is locally totally consistent, not locally partially consistent

Jacques Chester [10:03 AM]
the general ledger at the bank is locally totally consistent, not partially consistent

Greg Turnquist [10:03 AM]
But to your point, two entities can be /books and /authors. Create /books/1 and /authors/1, and then you can link them by putting /books/1 to /authors/1/books.

Jacques Chester [10:03 AM]
the ATM and the bank ledger are eventually consistent

Jacques Chester [10:03 AM]
in my case

Jacques Chester [10:03 AM]
I am talking to a ledger

Jacques Chester [10:03 AM]
it needs to be locally totally consistent when I complete the POST

Jacques Chester [10:04 AM]
otherwise I will need to implement the whole infrastructure to guarantee eventual consistency, which recreates the facilities already provided by the relational database

Jacques Chester [10:04 AM]
if I have dangling books and authors

Jacques Chester [10:04 AM]
people will make fun of me on twitter

Jacques Chester [10:04 AM]
if I have dangling transactions and allocations

Jacques Chester [10:04 AM]
people will get fired

Jacques Chester [10:04 AM]
or get charged

Jacques Chester [10:05 AM]
and get mocked on twitter

Jacques Chester [10:05 AM]
it has to be atomic, that's not a negotiable non-functional requirement

Jacques Chester [10:05 AM]
I understand where you're coming from

Jacques Chester [10:05 AM]
for example, if it was a purely summative task, eventual consistency would be fine

Jacques Chester [10:06 AM]
but it's essentially a general ledger update, it has to be an atomic update

Greg Turnquist [10:07 AM]
One approach is to implement a "shopping basket" approach where you create entities, but they the DB holds a predefined state, like "NOT_COMPLETED". Then a custom controller can link them togethe and move their state to COMPLETED.

Jacques Chester [10:07 AM]
Sure, but now I'm building an inner platform

Jacques Chester [10:08 AM]
you're asking me to develop my own transactional platform

Jacques Chester [10:08 AM]
... on top of a transactional platform

Greg Turnquist [10:08 AM]
Or configure the setter of the right domain object to shift the state to completed when the link is set.

Jacques Chester [10:08 AM]
sorry, I am being sarcastic, it's a bad habit

Greg Turnquist [10:08 AM]
I'm not aware of REST or Spring data REST having a "startTransaction"/"endTransaction" operation.

Jacques Chester [10:09 AM]
agreed, I am not proposing a begin/end semantic

Greg Turnquist [10:09 AM]
So I don't see how to demarcate that on the client side.

Jacques Chester [10:09 AM]
I would prefer that a POST or PUT to a subresource be a transactional step that updates the toplevel and the subresource together

Jacques Chester [10:09 AM]
because they are logically connected

Jacques Chester [10:09 AM]
it's a single operation

Greg Turnquist [10:09 AM]
Are you visualizing a JSON document with both the parent and the child's data being POST'd in one fell swoop?

Jacques Chester [10:10 AM]
so let's break it out

Jacques Chester [10:10 AM]
some background first

Jacques Chester [10:10 AM]
we have "transactions" and "allocations"

Jacques Chester [10:10 AM]
transaction is basically a bank statement line

Jacques Chester [10:10 AM]
"$100 paid to Amazon"

Jacques Chester [10:10 AM]
allocation is an accounting dept concept

Jacques Chester [10:11 AM]
"Of the $100 paid to Amazon, $50 is counted against the market dept budget, $50 against the shared service budget"

Jacques Chester [10:11 AM]
For POST

Jacques Chester [10:11 AM]
we see POST /transactions/1/allocations {some: json} as creating a new allocations subresource

Jacques Chester [10:11 AM]
taking transactions/1 as the linked parent resource

Jacques Chester [10:12 AM]
it's meaningless to have an allocation without an owning transaction

Greg Turnquist [10:12 AM]
Do you ever, ever, EVER refer to /allocations? Or is it an @Embedded resource?

Jacques Chester [10:12 AM]
It's never referred to independently

Jacques Chester [10:13 AM]
we get an /allocations endpoint because of how @RepositoryRestResource works now

Greg Turnquist [10:13 AM]
In typical REST, /transactions/1 would one resource, and /alloations/1, the canonical reference for the other resource. And /transactions/1/allocation would simply represent the link, not the canonical URI. So...I wonder if Spring Data REST would support defining two tables, but relating them via @Embedded. BTW, is the backend Spring/Java or something else?

Jacques Chester [10:14 AM]
Good question, I don't know how @Embedded would affect it

Jacques Chester [10:14 AM]
we're working on a proof-of-concept for updating a legacy app

Jacques Chester [10:14 AM]
we used all the latest toys

Greg Turnquist [10:14 AM]
A) @RepositoryRestResource is not required to export a repo. It's only to override defaults Which brings me to 2) If you put @RepositoryRestResource(exported=falsee), SDR will stop exporting it, and instead, inline allocations whenever you fetch a transaction.

Greg Turnquist [10:15 AM]
If you give me a couple minutes, I can try to slap together a sample app based no you rmodel using start.spring.io and Spring Data REST.

Jacques Chester [10:15 AM]
OK

Jacques Chester [10:15 AM]
I'll brb

Jacques Chester [10:21 AM]
ok, back when you are, we're going to carry on here on another task.

Greg Turnquist [10:22 AM]
Cool.

Greg Turnquist [11:20 AM]
I'm afraid after a lot of digging, I don't see a straight through answer.

Greg Turnquist [11:21 AM]
I took a crack at http://docs.spring.io/spring-data/jpa/docs/1.8.0.RELEASE/reference/html/#repositories.single-repository-behaviour, but it generated an error on startup.

Jacques Chester [11:32 AM]
hm

Jacques Chester [11:32 AM]
well, thanks for looking into it

Jacques Chester [11:33 AM]
we'll report based on the assumption that Spring Data REST won't pick up the transactional subresources semantic that we would prefer

Jacques Chester [11:33 AM]
meaning that for POST or PUT, we'd write our own controllers etc

Jacques Chester [11:33 AM]
and for GETs, either use the repo stuff or write our own, depending on fit

Greg Turnquist [11:34 AM]
Yeah. You can still leverage Spring Data JPA (or whatever) under the hood. And also expose things with Spring HATEOAS, so you can hold onto HAL. Just not the prebuilt ops from Spring data REST (yet).

Jacques Chester [11:35 AM]
Cool -- so we might be able to make our own repos that implement the hybrid behaviour behind the repository abstraction

Jacques Chester [11:35 AM]
which is what I personally would prefer

Jacques Chester [11:35 AM]
OK, we're off to lunch here

Greg Turnquist [11:35 AM]
See ya. Me too.

Jacques Chester [11:35 AM]
thanks for your help

---

1 votes, 5 watchers