Re: [OpenCoin-devel] Status update and protocol and implementation questions

[mathew r. <mat...@gm...>]

Okay. I'm working through the emails on this subject from newest to
oldest, only adding comments where they haven't already been resolved.

2008/1/21 Nils Toedtmann <ni...@op...>:
> Hi!
>
> What's the correct noun to the verb "redeem" - "redemption"?

Seems to be

> Am Montag, den 21.01.2008, 12:27 +0000 schrieb Joerg Baach:
>
> > > On Tue, 2008-01-15 at 16:38 +0100, Nils Toedtmann wrote:
> > >> On Sun, 2008-01-13 at 20:53 +0100, Joerg Baach wrote:
> > >>> On Sat, 2008-01-12 at 00:01 -0600, mathew ryden wrote:
> > >>>> - The DSDB does not unlock the coins if the time expires and
> > >>>>   they aren't spent
> > >>>> - The IssuerService does not change the status of the coins
> > >>>>   to spent when they are spent
> > >>> Why would it do that? I thought the coins get added to a list in
> > >>> the dsdb?
> > >> I guess Mathew that's what Mathew meant, right? When a coin gets
> > >> reported the first time to the DSDB it is put on the list and gets
> > >> marked "LOCKED". I read Mathew that the DSDB neither unlocks/forgets
> > >> the coin ater lock expiration nor changes the mark from LOCKED to
> > >> SPENT, yet.

Correct. At that time, the code did not prevent double spending the
same coins. It has been fixed since then.

> > >> BTW: The DSDB should forget, not unlock, right?

Aren't those the same things? Forgetting about a lock and unlocking on
a database where everything is inherently unlocked should be the same
thing, I figure.

> > >>>> - The code only supports one type of currency during a
> > >>>>   transaction between wallets
> > >>> Fine for now
> > >> ack

Does it make sense to support two different currencies in the same
wallet transaction? It makes things more ambiguous. For example, we
get the coins, and for some reason the IS refuses some when we attempt
a REDEEM_COINS_REQUEST. At that point, we may have one IS which has
accepted and another which has failed. How does the the consumer
wallet (Wallet Alice) continue from this point?

If, instead, we go the route of only allowing one currency, there are
no ambiguities. The IS rejects all of them and we continue on with
trying to fix the problem.

> > >>>> - A RedeemCoinsRequest may fail if the MintRequest isn't already
> > >>>>   in.
> > >>> ???
> > >> I read: when coins get exchanged (= redeemed against fresh coins), the
> > >> minting request must happen before the redeeming request - correct?

Not true by the protocol. Should we make it so a RCR must come after a MR?

> > >> What happens on final redeemtion (= not against fresh coins but
> > >> something else like RM (RealMoneyTM))?
> > >
> > > More on coin exchange below.
>
>
> > >>>> Now protocol questions:
> > >>>> 1) I have made the assumption (and plan to stick with it) that the
> > >>>> DSDB and IS are located at different addresses. It may even be true
> > >>>> that they are seperate entities! (Imagine the different ISs all
> > >>>> contract with the same DSDB service).
> > >>>>
> > >>>> 1a) For this to be viable, we need to add a service location field to
> > >>>> the DSDBKey. Should I go ahead and do that?
> > >>> To the key or the cdd?

To the DSDB Key.

> > >>
> > >> The DSDB is heavily linked to the IS as a LOCK may shift into a
> > >> redeemtion (and maybe additionally a minting) process. I do not see how
> > >> a wallet should contact a DSDB and a distinct IS during a lock-->redeem
> > >> +mint conversation.
> > >>
> > >> And DBDB requests which to not switch over into a redeemtion/exchange
> > >> will be extreemly rare.
> > >>
> > >> So the DSDB itself may be at a different location within the issuer
> > >> infrastructure, but that should not bother the wallets. The IS should
> > >> proxy DSDB requests.
> > >>
> > >> Or am i missing something? What exact (attack) scenario whould a
> > >> independant DSDB solve?

It would not solve any attack scenarios. It may however make sense for
the IS's to pool their resources together to make a global DSDB.
Separating the functionality would allow for this to happen. The
separation of the DSDB features and the IS features may also make a
more secure product, but I have nothing to back that up.

An attack on the separate DSDB and IS is that if the DSDB is available
and the IS is unavailable, the keys can be locked and the coins sent,
but can never be redeemed. I think the COINS_REJECT (or FAILURE,
whichever it is) covers that case though.

>
>
> > >>>> 1b) Supporting the multiple currencies using the same DSDB, I
> > >>>>     moved the signature to the additional signatures field.
> > >>>>     That way each CDD signs the DSDBKey
> > >>> a cdd signs a dsdb key? I thought an issuer would sign it? Does a
> > >>> cdd have a signing key?

Is the intention for the IS to use the same key for signing each CDD?

> > >>>> 1c) Does this mean that the DSDB itself should sign the
> > >>>>     certificate?
> > >> [omit 1b and 1c because 1a has to be clarified before]
>
>
> > >>>> 2) The DSDBKey has not_before and not_after fields. Does this mean
> > >>>> that all currencies that use this DSDB need to expire at that time?
> > >>>> Answer for 2) No.
> > >> true
> > >
> > >>>> the DSDBKey is only used for encrypting the serials
> > >>>> (and maybe depending on (1a), the location)
> > >>  ... and it is mayby used for singing answers, but that's still only for
> > >> trusted communication, so DSDB key could change without invalidatiing
> > >> any minting keys.

Agreed.

> > >>>> 3) When we LOCK_COINS_REQUEST and later REDEEM_COINS_REQUEST, the
> > >>>> coins have to be exactly the same, right? That is, we have the exact
> > >>>> same coins as the serials we locked, no less.
> > >>> Would say so
> > >> Good question. A wallet could redeem only a subset and let the locks for
> > >> the other coins expire (but why locking them then?). And a wallet could
> > >> try to redeem coins without locking them before (which means that the
> > >> wallet accepted them without DS check, which is dangerous).
> >
> > Isn't there the case of offline transactions, without ds check, usually
> > between trusted parties? In this case I would not need to do a lock -
> > however doing a lock beforehand also does not really hurt (besides not
> > being necessary).
>
> Yes, and there is at least one more case: when i want to
> refresh/re-split coins i once created myself, i do not need a lock. So
> it should not be mandatory to lock coins before redeemtion.
>
> But we could still require that *if* a list of coins is locked then this
> list can only get redeemed complete or not at all.

I agree with this. However, we do end up with an extraneous
transaction_id in the RedeemCoinsRequest. What value would we put in
it if we did not use the DSDB? I assume zero.

> > >> So it not really nessecary to insist on the same set of serials, but in
> > >> the real world it makes no sense for wallets to behave otherwise.
> > >>
> > >> The question is: which way is simpler? Let's do that.
> >
> > I would say that we don't need to enforce the same list. So it should
> > not be an error if I lock n coins, but only redeem n-1 coins.

The question becomes: what do you do with the leftover coins? Are they
unlocked or do they remain locked for the rest of the time?

> > > I thought it over again. Letting things open in specs leeds to
> > > incompatible inplementations. So i agree with Mathew: Although a payment
> > > scheme would not require that, we should insist on the same list of
> > > serials. At least unless someone finds a good argument not to do so.
> >
> > Reason a: another thing we need to check, enforce and handle.
> > Reason b: a split transaction of some sort - I want to accept all coins,
> >           and redeem some, and exchange some.
>
> I do not really have a strong attitude on this topic, maybe you are
> right that LOCK and REDEEM should be completely independent of each
> other, and that it's up to the wallet which coins get locked before
> redeemtion. Means also that our atoms are coins (and not lists of
> coins), at least when it comes to redeemtion.
>
> Mathew?  (hey, i feel a bit like a CNN reporter :-)

I'm trying to remember exactly why I wanted them in the same order. We
already require things to happen in the same order in other places.

Between wallets, BlankPresent and CoinsRedeem require the blanks and
the coins to be in the same order (I'm sure that's a requirement in a
document somewhere).

IIRC, the reason I was doing it between the Locking and the Redeeming
was for simplicity. I cannot come up for any other reasons at this
time.

*shrug* I'm happy to remove any requirement like that.

> > >>>> 4) MINTING_KEYS_FETCH_DENOMINATION returns the currently mintable
> > >>>> MintKey for that denomination, right (even if it expires in .5 seconds
> > >>>> seconds..)?
> > >>> Where are you in the protocol?
> > >
> > > Where the wallet asks the IS about the current minting keys to blind
> > > for:
> > >
> > >   <https://trac.opencoin.org/trac/opencoin/browser/trunk/standards/protocol.txt?rev=34#L79>
> >
> > Thanks, it was the additional s

Bringing up an additional point. We sometimes use plurals and
sometimes the singular. I think we should move everything to the
singular version.

> > >> Hmmm ... alternatives? It could return future key_ids as well. Client
> > >> will read date/time limits on keys an knows which one to use or if
> > >> better to wait a few minutes to not run into a minting key expiration.
> >
> > I think it should deliver future keys as well, returning
> >
> >  MINTING_KEY_PASS(keycertificate,[keycertificate...])
>
> Ack. What about the wallet asking
>
>   MINTING_KEY_FETCH_DENOMINATION(denomination, [deadline])
>
> expecting all yet published certs which validity has non-empty
> intersection with the period [now:$deadline].
>
> Btw: a wallet usually need more than one denomination. What about
>
>   MINTING_KEY_FETCH_DENOMINATION(list(denomination1, denomination2, ...), [deadline])

Okay. What about adding MINTING_KEY_FETCH_KEYID(list(keyid1, keyid2,
...) as well?

I'm happy with making all of these into list forms. We would also have
to change MINTING_KEY_FAILURE to return (list( (keyid/denomination,
reason), ...)


> > >>>> 5) protocol.txt states that the transaction_id for the DSDB locking
> > >>>> and the MintRequest is created using a two-party agreement. Is there
> > >>>> any reason it shouldn't just be a randomly generated number of some
> > >>>> length?
> > >>> I would have the feeling that would suffice. Nils?
> > >> I cannot recall any such reason (which does not mean there is none),
> > >> too. Wallet should just ask /dev/random (unguessable by mallory).
> >
> > Ok, so random number is it.

Agreed




> > >>>> 6) If the IS/CDD is setup (see 9) to require MintRequests to be
> > >>>>    sent to the IS prior to a RedeemCoinsRequest,
> > >
> > > I do not see a reason why an IS should require that? (What have i
> > > forgotten?)

Some IS's may not want to store value. If done this way, they only
hold the paid-for signatures, even if they haven't been computed then.
It's for the lawyers to be happy.

> > >>>>    should it be an error if we have not minted yet? Should the
> > >>>>    protocol for that requirement change
> > >>> I don't get the quiestion - if I read the protocol in the case of the
> > >>> RedeemCoinsRequest (RDC), MintRequest is sent along with the RCR.
> > >
> > > But asynchronous in two requests (by now), see (9)
> > >
> > >>> In exactly what case should it be an error?
> > >>>
> > >>>> to give the MintedBlanks right away during the RedeemCoins Request?
> > >>> Minting can take quite a while, so no, the coins should not be assumed
> > >>> to be given immediately.
> > >
> > > Usually not, but some issuers may support just-in-time-minting ("JITM")
> > > (e.g. if the tokens have no actual value).
> > >
> > > [see my answer to (9)]
>
>
> > >>>> 7) Does the opposite condition, the one where MintRequests are not
> > >>>> allowed until the RedeemCoinsRequest make sense? Should we implement
> > >>>> it?
> > >>> I thought the come together...
> > >
> > > Again, no (yet)
> > >
> > > [see my answer to (9)]
>
>
> > >>>> Protocol declarations:
> > > [Corrected from 7 to 8]
> > >>>> 8) We need some sort of handshaking, be it just something like
> > >>>>    "OpenCoin 1\n", "OpenCoin 1 Agree\n". The other allowed
> > >>>>    transaction
> > >>> Or you put it into the messages that you pass along. The whole thing
> > >>> started with more or less http like request/response pairs. The whole
> > >>> thing of having handshake etc. requires a stable session.
> > >>> Maybe I am lost and the protocol has advanced too far anyhow in the
> > >>> direction of keeping a stable connection. Then the proposal would be fine.
> > >
> > > Didn't we already agree that the transport layer below our protocol
> > > shall give us sessions? I am not sure ...
> >
> > Ok, for sockets we have some form of session anyhow (assuming that the
> > connection does not break in between). For HTTP we can use cookies.
> > So I guess it would be fine.
>
> And it's not really a disaster if the session dies a this point - just
> establish a new one and send the handshake again.
>
> BTW: Shall the handshake be mandatory? I vote 'yes': our protocol is
> very verbose anyway and not optimized on bandwidth at all, so adding
> this overhead will not hurt.

I think we need to have mandatory handshake handling. I am not against
mandatory handshakes.

> > > Anyway, i agree on having a handshake. That would be also a place to
> > > communicate information which changes from time to time (issuer would
> > > have to change CDD everytime), like "Prepaid only". "JITM supported",
> > > "JITM currently unavailable", "asynchronous exchange not supported".
> > >
> > > As these additional information are not bound to special transactions,
> > > this makes even sense if our protocol has no sessionsbut only
> > > request/response pairs.
> >
> > OK, so I propose a
> >
> >       HANDSHAKE(protocol version, subprotocol)
> >
> > with subprotocol being something like "sendMoneyProtocol", indicating
> > what we would like to do.
>
> Hmmm ... what shall "subprotocol" that be good for? The wallet's next
> statement will make that clear anyway.
>
> > The other side can response with either
> >
> >       HANDSHAKE_ACCEPT(protocol version, subprotocol, [data...])
> >       HANSHAKE_REJECT(reason)
> >
> > Generally speaking right now we never seem to exchange key-value pairs
> > in the messages, which I would actually would find quite usefull.
> >
> > E.g. the Issuer could then return something like
> >
> > jitm=1,prepaid_only=1,someotherstuff=foo
> >
> > with that we would not rely on the order of things. A bit like the
> > headers in the http protocol.
> >
> > What do you think?
>
> I almost agree. I wonder if we should combine the ESMTP way ...
>
>   (client connects to server)
>                                   220 trinity.takatukaland.de ESMTP TakaTukaPost
>   EHLO laptop.nt.eonis.de
>                                   250-trinity.takatukaland.de
>                                   250-PIPELINING
>                                   250-SIZE 30000000
>                                   250-ETRN
>                                   250-STARTTLS
>                                   250-AUTH PLAIN LOGIN
>                                   250 8BITMIME
>
>   (you see a nice trick here: a hyphen after the response code indicates
>    "there are still more lines to come for this response")
>
> ... with the X.509 way: There are protocol extensions (like JITM), which
> were not necessarily part of the original protocol, or which are not
> supported by all participants. Some are 'critical', which means "if you
> do not know this extension, then quit now" ("my way or the highway" ;)
> For example
>
>   HANDSHAKE_ACCEPT(protocol version, jitm PREPAIDONLY maxcoins=100)
>
> uppercase signalling a critical extension. On the other side, there is
> the CDD for long-term information. So i think: if JITM is supported by
> an issuer in general, it should be mentioned in the CDD. Protocol
> extensions should only be used for
>
> * short-term variations of the CDD (like "jitm currently not
>    available")
>
> * properties which are not properties of the currency or issuer but of
>    its issuer service itself (like ... encodings, compression?)

I think we need to describe the CDD features and limitations in the
handshake. This allows the currency to last longer. If the laws change
in the area and the IS needs to change from jitm to a 3-minute delay
with relaying to a third party (not that I expect laws to get that
bad), the worth of the currency and all coins is not affected.

We don't have any mechanisms for changing the CDD (Although we can
sign the CDD again....)

Hmm. Maybe one of the handshake commands refreshes the CDD?
Maybe there is a command to get the CDD from the IS?

Okay. I'm fine with putting information in the CDD, but we need to
have a system to update the CDD (And downloading the CDD every time
doesn't make sense)

> > > [Corrected from 8 to 9]
> > >>>> 9) Right now the protocol allows for either forcing mint requests
> > >>>> prior to redeem coins requests, or allows them before or after the
> > >>>> RedeemCoinsRequest. This information needs to be given to the wallet
> > >>>> somehow. One way is to have that part of the handshaking (see 8).
> > >>> Still don't get the question.
> > >
> > > I pound the exchange part for quite some time now. This is my current
> > > state of thoughts on that (may change next minute):
> > >
> > > * Some issuers may prefer to only accept a MINT_REQUEST if that is
> > > already paid (whether paid with OpenCoins or RealMoney (RM) does not
> > > matter), so there has to be either an CDD-option or a handshake option
> > > "PrepaidOnly".
> > >
> > > * Some issuers may support just-in-time-minting/JITM, our protocol
> > > should do that, too.
> > >
> > > * We should make MINT_REQUEST more similar to REDEEM_COINS_REQUEST,
> > > splitting "request_id" into "transaction_id" (identifying this list of
> > > blinds) and a "target" (identifying an external payment):
> > >
> > >   MINT_REQUEST( transaction_id, target, list( (blind1.key_id, blind1), ... ) )
> >
> > Fine with me

More on this later.

> > >
> > > transaction_id=0 means mandatory JITM (which may get rejected). Issuer
> > > may accept
> >
> > may accept with...
> >
> > >
> > >   MINT_ACCEPT ( transaction_id, list( signature_of_blind1, ...) )  # JITM

Wouldn't it make more sense to just send a FetchMintedAccept in this
case? It seems better to me to overload the orders of the messages
than their meanings. It would use the transaction_id of 0 for the
JITM. More on this later.

> > >   MINT_ACCEPT ( transaction_id )                                   # delayed minting
> > >
> > > or reject. There are additional reject reasons:
> > >
> > >   "JITM not sopported"
> > >   "JITM currently unavailable",
> >
> > Looks good to me
> >
> >
> > > * Yes, you are right (and i think Joerg once voted for this, too):
> >
> > Ack
> >
> > > besides asynchronous REDEEM_COINS_REQUEST+MINT_REQUEST there should be
> > > another way to refresh coins with a single request like
> > >
> > >   EXCHANGE_COINS_REQUEST(transaction_id, list(coin1, ...), list( (blind1.key_id, blind1), ... )
> >
> > I assume thats a very often used transaction. I like it
>
> Me too. I originally preferred the REDEEM_COINS_REQUEST+MINT_REQUEST way
> because it falls from the sky without requiring YetAnotherRequestType,
> but i agree that we really want a synchronous exchange, too.
>
>
> > > If transaction_id=0, the wallet insists on JITM. Possible answers:
> > >
> > >   EXCHANGE_COINS_ACCEPT( transaction_id, list(signature_of_blind1, ...) )   # JITM
> > >   EXCHANGE_COINS_ACCEPT( transaction_id)                                    # delayed minting
> > >   EXCHANGE_COINS_REJECT( list( (blind1.key_id, "Reason1"), ... ) )

Just a quick change. EXCHANGE_COINS_REJECT( list( (blind1.key_id,
blind1, "Reason1"), ...)

Actually, the passing of actual blind from the issuer in MINT_REJECT
seems to be missing as well.

It may be possible in some signing methods to have individual blinds
which are not blindable.

I would also like to use FETCH_MINTED_ACCEPT in place of
EXCHANGE_COINS_ACCEPT with JITM, otherwise MINT_ACCEPT.

It does seem to be getting messy though. Hrm. :\

> > > Does that make sense to you? If you agree i change our Protocol.txt
> > > accordingly. (I should learn how to branch svn).
> >
> > +1
> >
> > Also: change it in trunk. We still have the old revisions. As we are
> > working together on the protocol anyhow, we should have one place to
> > refer to.
>
> ok
>
>
>
> > >>>> Request for clarification:
> > >>>> Can someone go through all the ways we can have a
> > >>>> MintRequest/RedeemCoinsRequest/FetchMintedAccept? (See 6 & 7). Does it
> > >>>> make sense to allow all of them? (So three options, I think)
> > >
> > > (A) "initial buy": Buying OpenCoins, paying with RealMoney or
> > >    something else
> > >
> > >       MINT_REQUEST(transaction_id, target, list( (blind1.key_id, blind1), ... ) )
> > >
> > >    May get rejected if
> > >
> > >     * transaction_id=0 but JITM is not supported/available;
> > >     * issuer requires "Prepaid only" but $target is not paid yet
> > >
> > >
> > > (B) "exchange/refresh": Buying OpenCoins with OpenCoins.
> > > (B1) asynchronious (this is just A+C, so we get it for free):
> > >
> > >       REDEEM_COINS_REQUEST( transaction_id, EXCHANGE, ...)
> > >       MINT_REQUEST(         transaction_id, EXCHANGE, ... )
> > >
> > >    This order should always be accepted (or??). If issuer is "Prepaid
> > >    only" he rejects the order MINT_REQUEST, REDEEM_COINS_REQUEST.

I think there are times when we cannot follow this method. This method
may be viewed as a credit institution since it stores value.

That is, I believe some issuers may want REDEEM_COINS_REQUESTs to
occur after the MINT_REQUESTs, and some others may always want the
opposite.

Actually, the only reason for supporting the REDEEM_COINS_REQUEST
before the MINT_REQUEST that I can see is to prevent the overhead of
minting.

> > > (B2) synchronous
> > >
> > >       EXCHANGE_COINS_REQUEST(transaction_id, list(coin1, ...), list( (blind1.key_id, blind1), ... )
> > >
> > >
> > > (C) "final redeemtion": Selling OpenCoins, getting something else.
> > >
> > >       REDEEM_COINS_REQUEST(transaction_id, target, ...)
> > >
> > >
> > > There is some inconsitency here: with (B1), the $transaction_id cannot
> > > be set to zero, so the wallet cannot insist on JITM here. We could
> > > circumvent this it we signal mandatory JITM not via $transaction_id=0
> > > but {whatever}.
> >
> > I think its ok to leave it as it is. The (small) inconsistency could be
> > allowed.

I must be missing the inconsistency (or I may have accidently fixed it
in my description below):

First off, requests without JITM:
MINT_REQUEST(request_id, list((blind.key_id, blind))) is sent. The
request_id refers directly to the blinds.
REDEEM_COINS_REQUEST(transaction_id, target, list((coin))) is sent.
The target refers to the request_id of the MINT_REQUEST.

(These can be in either order)

Now, to make it work with JITM, we add a target field to the
MINT_REQUEST. In the JITM-less version, we just make the target the
same as the request_id.

We have two different ways we can do JITM.

1) REDEEM_COINS_REQUEST before MINT_REQUEST:
REDEEM_COINS_REQUEST(transaction_id, target, list((coin))) is sent.
The target refers to the MINT_REQUEST's request_id

MINT_REQUEST(request_id, target, list((blind.key_id, blind))) is sent
with a target of 0.
A target of 0 forces it to be a JITM. It has MINT_REJECTs based off if
JITM is disabled.

FETCH_MINTED_ACCEPT (or the overloaded MINT_ACCEPT) is returned.

2) MINT_REQUEST before REDEEM_COINS_REQUEST:
MINT_REQUEST(request_id, target, list((blind.key_id, blind))) is sent
with a target of 0.
REDEEM_COINS_REQUEST(transaction_id, target, list((coin))) is sent.

Since the target of the Mint Request was 0, the issuer sends a
FETCH_MINTED_ACCEPT (or an undefined overloaded REDEEM_COINS_ACCEPT).

It works in both cases but I seem to be using the target only as a
JITM identifier.


I read through the reasons for breaking up the MINT_REQUEST
transaction_id into multiple pieces, but I just don't see the reason.
The entire system works with it just being a single field. I don't see
what we gain by breaking it into two different fields since the target
seems useless to me as anything other than JITM marker.
You had said it could be used to show the payment of the Minting, but
I don't see a point in making and remembering another number.

Okay. Now a brand new protocol question (which I forgot to ask):

We have all these base64 random numbers. Right now I'm using (I think)
128 bit random numbers for them. Should the lengths of the serial,
minting request_id, and redemption transaction_id be standardized
somewhere?

Sorry about my absence. I lost the internet a day and a half before
going cross country for a wedding, just to find out that they had
somehow gotten an incredibly odd network configuration to work, but it
was not going to allow me access. Did you know you can hook a
cable-modem into one of the outputs of a wireless router, and some
(but not all) wireless clients may actually talk to the dhcp server of
the ISP? I didn't.

-Mathew