Friday, August 10, 2012

A Software Architect's View of the Design of Double Entry Paper Accounting Systems

Taking a brief break from the computer side for now, I figured it would be worth describing the basic design considerations of double entry.  This post is the result of my work studying history and anthropology far more than working on LedgerSMB but of course working on the software played a role too.  Most of what is presented here is my own original research.

Also I am sure that accounting students or those who have studied accounting in school will find some aspects of this challenging since I explore some areas that accountants are not taught about and because, as a history buff, I refuse to believe that certain designations are arbitrary.

I think that the apparent evolution of double entry accounting shows a number of potential approaches for dealing with the very difficult distributed transaction issues of today, not so much in what to do but where to look for answers.

The Tally Stick System and the Origins of Double Entry

In medieval Europe there were two basic economic constraints.  First most people could not read or write or perform arithmetic on paper, which limited the forms of exchange possible.  Secondly there was a perpetual shortage of currency which meant that official money was not really possible to use money as a simple medium of exchange.  These problems were resolved by the development of the split tally stick.  The split tally stick evolved through the Middle Ages into a very highly developed system and was still in active use in some places into the 19th century.  Ironically it was an effort to destroy these sticks held by the British Government in 1836 which burned down both houses of parliament.

At its fully fully developed form, a split tally stick consisted of a stick, usually of hazel, which had notches carved in it.  The stick would be split length wise and one side would be cut short.  The long side would be called the stock (trunk) and represented the debt.  It would be held by the creditor.  The short side, called the foil ('leaf') would follow the loan.

The creditor, at the allotted day, could show up and present his stock and demand payment.  If the creditor tried to pad the debt by adding more notches, this would be immediately apparent, and there was no way to erase notches on the other side.  Additionally it was immediately apparent if the sides were from different tally sticks.

As the Middle Ages progressed, literacy became somewhat more widespread.  From the days of the Merovingian king, Charles Martel,  through the "Carolingian Renaissance" in to the high Middle Ages, literacy began to become more open beginning with kings and eventually being available to the children of wealthy merchants.  With literacy came basic knowledge of arithmetic and eventually algebra, and the efforts to track tally sticks on paper may have given rise to double entry systems of bookkeeping.

The first step of course is a journal.  Here the tally sticks could be recorded.  The bigger stocks (like bigger numbers) would be reported on the left, while the smaller foils would be reported on the right.  One could then get a quick breakdown of one's position in terms of debt collectable vs debt owing.  Within a few hundred years, this develops into general ledgers and the full double entry accounting system that has not changed in its outlines since Luca Pacioli wrote about it in the 15th century.

However it is worth noting that split tally sticks are inherently double entry in the sense that every debit has a corresponding credit, and they are inherently accrual since income and formal payment could not be as well correlated as could invoice and income.  Moreover the rules for stock and foil appear to have tracked the rules for debit and credit today.

It is finally worth noting that even in Pacioli's day, and even a hundred years later, tally sticks were in common use in this way.  Therefore, it seems to me that Pacioli himself was probably familiar with these and so his use of Latin terms corresponds almost certainly to the tally stick approach.


Business as Owning Nothing for Itself

Many of the principles I have found while investigating the origins of double entry accounting are challenging to us today and force us to think about businesses differently.  Exploring the development of designs of older functional systems also better prepares us to think about design ourselves in areas of software where we are building complex functional systems today.

In a double entry accounting system, a business owns nothing for itself.  Everything it owns, it owes to its owners.  This is why the books always balance and why debits always equal credits.  Corporations are a legal fiction which post-date the development of double-entry accounting systems but even they don't own things for themselves.  Their equity is owned entirely by their owners.  A corporation owns things only on behalf of its shareholders.  Limited liability only means that the effective equity cannot drop below 0 and a corporation has no possessory interest distinct from those of its stockholders.

This understanding is also behind the fundamental accounting equation that assets - liabilities = equity.

As long as the books are guaranteed to balance, then it is possible to detect errors by balancing the books (using a trial balance, as Pacioli suggests).  This is only possible if the intrinsic net worth of the business is always 0 (when we talk about the net worth of the business, we are talking about the extrinsic net worth, namely the equity balance, see below, but this is actually the net balance of debt owed by the business to its owners--- while an individual may on balance be worth a million dollars, this means something very different than if a business is worth a million dollars because the business, unlike the individual, owes that money to someone).

While stock and foil tally sticks were originally used to track debt, Henry I of England required that they be used for receipts for taxes in the 11th century.  The process followed the pre-existing use, where the stock follows the person who has given money, while the foil was retained by the exchequer as a receipt of money retained.  The continued use of this sort suggests that the basic principles of double entry accounting were  already known to some extent.  If tax liability is a debt owed, then the payment is a debit and one receives a stock, while the foil follows the money given.

It is my belief that the basic principles of double entry accounting were explored first with tally sticks and later, as literacy became somewhat more widespread, on paper.

What Accounting Systems are Designed to Do

Accounting systems are designed to do one thing only, to track who owes what to whom.  Even tracking of current assets is a part of that since all assets of a business are effectively owed to the owners.

This may sound overly simplistic but the anthropological assessment on the origin of money is that money itself arises as a way to quantify debt.  Debt pre-exists money systems, and all economies are powered by debt, and this is particularly noticeable in gift economies where debt, in the form of honor, is the primary currency.    Everywhere debt precedes payment, and everywhere debt precedes money.  Therefore not only is accrual-basis accounting better in terms of reporting, but it better represents what actually is happening economically.

Debits and Credits

In the 15th Century, Luca Pacioli wrote his now-famous book on arithmetic, including a section on double-entry accounting.  Pacioli did not invent the system.  Instead his description is that of the Venetian system which was already in use at the time.  In general, the design is indicative of accrual-based accounting however in part because assets include debts owed to the business.  This actually is clearer when looking at the Latin terms Pacioli uses in his descriptions and how these fit together,  Namely he uses the terms "debit" and "credit" to refer to financial units which are clearer in Latin than they are in English and much of our accounting terminology derives from his work.

Pacioli's use of debit and credit are specifically of distinct concepts, and when every accounting student is taught these are arbitrary, every accounting student is taught wrong.

In Latin, debit refers to something which is owed and indeed it leads to our modern English word debt (via Old French).  The word derives from early Latin roots meaning "to take away something you have" and so it denotes a loss of a possessory interest in something.

A credit is the opposite side of a debit.  A debit is something owed.  A credit (from credere, to believe or trust, related to "creed" in Modern English) is something entrusted to someone else or loaned to them.  This term denotes a continued possessory interest in something, even as it is entrusted to someone else.  So we can most simply translate debit and credit as debt and investment or loan, respectively.  Being opposites, a credit abates a debit and vice versa.

Also being opposite sides of the transaction every transaction inherently balances.  One person's debit must by nature correspond to someone else's credit.  By entering everything from the perspective of the counterparty, whether customer, vendor, or owner, the books will be guaranteed to balance, and this allows one to detect errors because the total intrinsic value of the business will remain 0.

For this reason both debits and credits have two distinct meanings.  One is to off-set the other (paying back a loan is a debit against the corresponding credit), and the other is to represent debts (debits) and assets entrusted to the business but possessed by others (credits).  Every type of account furthermore has a specific type of counter-party but these fall into two categories:  owners (equity, and change-in-equity accounts, namely accounts for tracking income and expenses) and non-owners (assets and liabilities).

From this point we can derive the normal balance of every account because we understand the basic structure and functions of the system:

  • Asset accounts track debt and debt payments by those who owe money to the business.  Therefore they use debits for positive balances.  Being debits, they can be used to pay off loans made to the business (credits) or investors (investments are credits).  The perspective is that of the debtor.
  • Liability accounts track loans and loan payments to those the business owes money to.  Therefore they use credits for positive balances.  The perspective is that of the lender.
  • Equity accounts track investments in a business which reflect the value of the business to the owners.  Therefore they use credits for positive balances.  The perspective is that of the owner.
  • Income accounts track positive changes in equity.  Therefore they use credits for positive balances.  The perspective is that of the owner.
  • Expense accounts track negative changes in equity.   Therefore they use debits for positive balances.  The perspective is that of the owner.

What Software Engineers can Learn from Accounting Systems

Most of the time, those of us who design and write software find that our approaches are relatively orderly but fragile.  Aspects of a system fail to support each other, and we basically add complexity to the system in order to hold it together.  Highly engineered systems are thus brittle, or to the extent that they are not, require layer upon layer of complexity in order to keep working.  These systems often cannot continue to be maintained properly once people have forgotten why certain design decisions were made in the first place.

In contrast, double entry accounting systems (particularly accrual-based systems), which probably arose organically during the Late Middle Ages until they became important enough for Pacioli to write about, is a highly evolved system.  The overall approach, while difficult to grasp at first, can be maintained without any understanding of the reasons behind the design decisions.  Accounting systems have further evolved from Pacioli's day, presumably through the same process they evolved before then.  People understand, in general terms, principles required to get meaningful information out of the system, are confronted by new problems, and respond by experimenting and sharing results, until new approaches take root.

One thing we as engineers can do is look to some of these highly evolved systems, and how they change over time, and recognize that they show us what may be a better way to create highly robust systems that tolerate and detect errors well.  In double entry accounting, we look to owners 'interest in the business vs the business's interest in everyone else's assets and make sure they are equal.  This adds redundancy in entering information but it also adds richness in reporting that is not possible otherwise.  Perhaps there are opportunities for things like this elsewhere.

Here, with double entry accounting, the perspective chosen, namely the value of a business to itself, is an easy one to check.  Such a business will inherently have no value to itself.  Choice of perspective makes some difficult problems (like catching data entry errors when tracking money) relatively simple to catch, locate, and correct.   Sometimes the most elegant solutions are not in what you do but how you look at things.  This is very clear when it comes to this specific sort of system.

The real challenge going forward in my view is how we look at distributed systems and what perspectives we can find which make these problems elegantly solvable.  As with double entry accounting, this will probably have to arise from looking at pre-computing methods, for example petty cash management as a basis for a guarantee of eventual consistency in a distributed transaction.  Current approaches like those adopted by the NoSQL community don't get us there.  Approaches that draw from our experiences doing non-distributed transactions well as well as paper forms of distributed transactions may, however provide something a lot more robust and usable.

26 comments:

  1. "the anthropological assessment on the origin of money is that money itself arises as a way to quantify debt. Debt pre-exists money systems, and all economies are powered by debt"

    I'm not sure where you got that. In my book, money arises and is a medium of exchange, and economies are powered by savings.

    ReplyDelete
    Replies
    1. Perhaps you could choose a book slightly better based in the evidence then at hand and not contradicted by events since. I recommend the works of John Kenneth Galbraith as examples of these.

      Delete
    2. Wrong on both counts.

      "Money" arose (and not only, or even initially, gold) to simplify barter. Barter is still used, especially in many of the black markets planet wide. "Money" only obscures value, since some (hint, hint) conflate "money" with production.

      Economies are not powered by savings. Another Right Wing shibboleth. "Progress" comes from technological increase. We don't live as we do today, compared to 1800, because of MONEY. Why would anyone make that leap? This change in life was driven by technological progress, particularly in agriculture. "Money" had nothing to do with it. Various -ologists have all the evidence one needs to understand how paleolithic man created cities (with many smaller settlements along the way): food surpluses, due to technology, allowed for specialization of work effort. Humans were then able to develop special skills (with which they traded for food and other necessities), and technological progress accelerated. "Money" had nothing to do with it. Unless one defines "Money" as just excess per capita food production; and that ain't gold.

      "Money" didn't cause smart people to be smart and discover new discoveries. Some, yes, then exploited some of those discoveries solely for pelf; more often than not, not the actual inventor of course. Einstein didn't get smart in pursuit of money. He was just smart.

      And to answer the following comment here: the Austrians suffer from various mental diseases.

      Delete
    3. Gift economies run on a currency of debt which is termed honor, correct? Barter only arises with regard to outsiders who one will not see frequently.

      I am thinking in particular of books like http://www.amazon.com/Debt-The-First-000-Years/dp/1933633867 but maybe I was wrong about whether it was largely a consensus view.

      The basic problem you have to get over in asserting that money is barter is the fact that people engaged in longer-term efforts don't have things to exchange most of the time. Consequently they give things away in order to get a reputation that encourages others to give to them. Even in societies with money, where money is in short supply, gift economies can and do thrive. For example, my historical leanings immediately lead me to suggest Anglo-Saxon England and the gifts by lord to follower.

      Delete
    4. Hello Robert,

      Saying that money arose to "simplify barter" is the same as saying that money is a medium of exchange (barter = exchanging goods or services).

      When economists talk about savings that includes technological advances. To quote Rothbard: "There is another unique type of factor of production that is indispensable in every stage of every production process. This is the 'technological idea' of how to proceed from one stage to an­other and finally to arrive at the desired consumers' good.". But the technological recipes aren't enough. Someone has to save, i.e., not consume all that they gather or produce, so that they can take advantage of those recipes.

      Also, "powered" has two connotations. Undoubtedly, technology makes progress possible. But savings (e.g., capital goods, including the technological knowledge) are what fuels an economy. Economies don't grow because people borrow or spend money.

      Have you read Rothbard or Mises? If you haven't, on what do you base your mental disease diagnosis?

      Delete
    5. Joe: But what are you saving?

      Money is a way to quantify and transfer debt. If you have savings, then you are basically saving favors that other people owe you. Now, this is transferable using money as a medium of exchange. Money is thus basically a debt voucher, and this is the case even where money is in short supply. For example, in the Middle Ages it wasn't that uncommon for artisans to create stock-and-foil vouchers for their services, let these vouchers circulate on market day as "money" and then to settle the accounts between themselves. Again debt is owing people goods and services. Wealth is other people owing you goods and services. Money is a medium of exchange regarding this debt.

      It isn't the only possible system of course. In ancient Ireland, there were two equivalent systems of money in use at the same time: silver (which functioned as a transfer of debt) and cattle (which functioned as a means of production and hence a productive asset).

      The only real assets not directly tied to debts economically speaking are productive ones, namely land, livestock, and tools. Capital = transferred debt and therefore is as much an asset as the balance of accounts receivable. The Irish system used both at a fixed exchange rate (so many ounces of silver for a cow) but even here the idea is you pay off debt by giving up an asset. The debt precedes the payment (otherwise there is no requirement to pay).

      Delete
    6. Money is indeed used to quantify and transfer debt, but money did not arise for that purpose. Instead it came about to facilitate trade: a chicken farmer wanted to buy shoes but the cobbler wanted potatoes instead of eggs. Coins (currency, money) allowed trades to happen because the farmer and the cobbler could agree on a certain number of coins as the price to be paid for the shoes, and being divisible (and standardized and accepted by others) the cobbler was confident he could use some of those to acquire the potatoes later.

      Savings in economics is unrelated to debt or money. A hunter gatherer, for example, saves by not eating all the berries or fish she has collected or trapped. Eventually she realizes she can collect more berries or get more fish by making some tools. Those tools are capital (or capital goods if you will), because they can be used to improve what economists call the production process (the hunting and gathering in this case). By being able to produce more with the better tools, the hunter gatherer can feed more people in her family and by saving what they don't consume, they can start to trade with other families, and thus "grow" the economy. Although this was presented in terms of a primitive economy, the principles still apply.

      Wealth is not based on people owing you goods and services. Wealth is the accumulation of savings. If BillG buys a mansion or a plane, and pays cash for them, he owns them outright and nobody owes him anything. If you buy a TV, it doesn't really matter whether you paid cash for it or you charged it, you still own it, it's part of your net worth, your wealth. If you hadn't saved enough before your purchase, then you'll have work, trade or otherwise acquire funds to settle your debt, but that is an entirely separate matter.

      Delete
    7. Joe: The basic issue though is that money (as structured currency) did arise for this purpose, namely to address and transfer (and even create, through taxes of specified currency) debt.

      Suppose I am a cabbage farmer. Cabbage will only be harvested a couple times a year and it doesn't have a huge shelf life. I will have a huge payoff but I can't keep goods around for barter. So what do I do in the absence of money? I give the cabbage away. I track who I gave it to. I go to them later and ask for favors. That's debt, plain and simple, and it predates money because in any society, even hunter gatherer societies, a significant portion of the population is involved activities that have occasional but high payoffs (consider the horse-hunting of the Botai-Tersek people for example). Absent debt you don't *have* a medium of exchange that works, unless you want to trade for my month-old horse roasts.

      The problem with debt of course is that it creates future obligations and absent a way to quantify them, you have a problem. If I give you horse meat and later come to you and ask for pottery, how do we determine what is a fair exchange? In a gift economy, reputation accomplishes this and if you give me more than was fair value people will back you when you come to ask for more meat. And so the economy runs, quite literally on debt.

      Official currency arises to fill two roles. The first is to quantify and transfer debt. The second (and somewhat sinister function) is to *create* debt through taxes with the idea that the debt can be worked off in service to the government, particularly the military. This was a huge function of early money systems. If you can create a population that is in debt to the government, you have an army ready to raise whenever you need them.

      In places like Republican Iceland that never had a very centralized government currency took the form of either useful goods (ells of homespun cloth) or medium for external trade (hacksilver). The later is more clearly a transfer of debt and second is more clearly a common medium of exchange based on a commodity.

      It is true that in some societies livestock was used as currency, but I don't think you can reduce this to barter any more than you can reduce the use of set lengths of homespun cloth as civil fines in republican Iceland as barter. Often fines are set specifically in legal codes in the currency whether this is productive-good based or not.

      The final point is that debt is found as a central religious point in virtually all mythologies and religions in all ages. These are clearly found in the Norse cosmology (my own original research), Greek Mythology (see F. M. Cornford's "From Religion to Philosophy"), and many others (again Debt: The First 5000 Years has a decent summary here). Debt, unlike money, is a universal aspect of the human experience. It is possible to go your life without ever touching official currency. It is impossible to go your life without ever being indebted to someone else.

      Delete
    8. This comment has been removed by the author.

      Delete
    9. It looks like you've been very influenced by David Graeber. For some contrast, I would suggest reading this and this. [previous comment removed because one link was cut-off]

      Delete
    10. Regarding Graeber, the major thing is he gave voice to something very different from my own studies, which have included medieval financial instruments and approaches to debt in Christianity, Norse Paganism, Judaism, and Hellenic Paganism. One of the huge influences on my ideas in this area was "From Religion to Philosophy" by F. M. Cornford, particularly his look at Moira of Homer and Hesiod and how this manifested in partition and debt in the ideas of early Milesian philosophers, particularly Anaximander.

      Prior to reading Graeber, I had already come up with a surprising set of conclusions relating to the universality of debt. I could provide you with a copy of my paper (somewhat inspired by Cornford's analysis but also relatively different) on the relationship between fate, time, and debt in the pagan Norse poem Voluspa (and expanding in the view of fate to other poems). Of course there is some reason to think the Greek myths inspired the Norse ones here but there are remarkably few formal connections not found in other IE groups, and there are a lot of formal differences of a sort similar to the differences between, for example, the Irish and Norse views, so I think these are independent developments. The use of debt terminology there is really fascinating there because life itself is treated as a primal loan which is repaid in death.

      However two basic points. The language of accounting is the language of debt. That cannot be avoided. 'Debit' and 'credit' are Latin terms about debt, and this is far more the case than the terms for the parts of the tally sticks used to track them. In Pacioli's time, then, it is very safe to say that cash was tracked as a debt equivalent rather than the other way around (the way we think of it today).

      Icelandic currency systems were informal and based either on weight of silver or lengths (ells) of home-woven cloth (early Irish currency was also either weight of silver or cattle). Neither of these were coin-based in the same sense of Rome or medieval France, but it wouldn't be fair to call them barter because they were official currencies used for things like payment of civil judgements to those who were wronged. Moreover these were systems formalized in law, in the way a currency system usually is (the Irish had a fixed conversion ratio between silver and cattle, for example). It's not money perhaps in the way you mean it but it is certainly not informal barter either. I think it *is* money on a grass-roots level though (and markets were regulated in these economies through relatively complex customs involving professional lawyers, btw).

      Delete
    11. Also a point where I think that the interviews with Graeber miss a point. What Graeber is saying is that coinage replaces the gift economy rather than barter. We do see barter with strangers in gift economies. If I am never going to see you again, I am not going to give you cabbage in exchange for some promise of a future favor. I need something now. That's where barter comes in.

      I think the truth is somewhat more complex than Graeber suggests. In places like early Ireland and early Iceland for example, homespun cloth and hacksilver did function as a medium of exchange, and homespun cloth existed also as a material for the production of physical necessities. One could argue that this is the sort of currency that arose through the independent actions o self-interested individuals in the standard story, but it ends up being entirely based on useful commodities (cloth, cattle), and they too appear to arise to prominence through the effort to quantify certain kinds of debt, namely civil and criminal penalties in the Irish and Icelandic systems. There is room for grass-roots development of money systems (seemingly always based on useful commodities such as livestock or cloth), but they too arise from the same needs. The early idea of justice really was that if you do someone wrong, you incur a debt to them (this is universal). This sort of debt is the sort which is most in need of quantification and gives rise to some sort of formalized currency at least in the cultures I have directly looked at which do not have Graeber's top-down systems.

      Delete
    12. One final point on this. The second link I found very interesting. The idea of a fixed equivalent between a commodity and silver is found in Ireland and Iceland as well (Iceland it is to cloth, and in Ireland it is to cattle). One thing that is in common regarding all these systems is that this is an early monetary system. I would define a monetary system as a series of instruments for paying debt where there are fixed equivalents between the instruments. If we have a system where everyone agrees that a cow is worth ten goats, and I have a choice to pay you two cows or twenty goats, that is a monetary system rather than a barter system. The same holds regardless of what the equivalents are, whether it is silver to grain, silver to cloth, or silver to cattle, stamped iron bars, or whatever.

      On a functional level this seems to arise from non-state actors and at least in that aspect seems to be in line with Menger's analysis. My knowledge of Mesopotamian temples and their economies is not as great as perhaps that of others, but my knowledge of early Ireland and Iceland is pretty strong. Iceland is perhaps the most interesting because, like the system described in the Mesopotamian Temples, the connected commodity was universally consumed, but unlike the Mesopotamian Temples, it was also universally produced (virtually all women spun and wove).

      Additionally in both the Irish and Icelandic systems, the state did not actually monopolize violence. Rather there was an organic oral-formulaic legal tradition for settling disputes over violence and the perpetrator of a violent act could generally pay off the victim and have that be the end of the matter. Jesse Byock's excellent work in this area has shown that blood feuds were actually important instruments of social regulation and stability in Icelandic society (Icelandic society had a legislature and a judiciary, but the executive branch was, quite literally, the people, and the law most certainly did not disfavor self-help). Similar systems were found in early Ireland and in pre-Solon Athens -- by my reading it wasn't really until Solon that the state fully monopolized the use of violence. The idea that markets and money emerge from top-down direction is an area where I think that Graeber makes a serious error. Decentralized processes can result in these things just as often, though they don't result in creating debt through taxes per se (even the so-called taxes of the so-called chieftains of Iceland are better understood as fees for services). The problem though is that in these decentralized processes it is hard to differentiate state from non-state actors. Is the Brehon of Ireland or the Godhi (usually translated as "chieftain" but a better translation might be "attorney") in Iceland a state actor? How do we define state? Does medieval/republican Iceland even count as a state given that they had no executive functions in their government?

      However in these two societies at least, money in the sense of financially tied instruments emerges primarily to address what we might think of as lawsuits. Everyday goods are usually distributed in a gift economy, and this is how important exchanges are handled as well.

      Delete
  2. I would recommend the works of Ludwig von Mises, Murray Rothbard or various other Austrian School economists. Or perhaps read the Australian Steve Keen (http://www.debtdeflation.com/blogs/), Mish Shedlock (http://globaleconomicanalysis.blogspot.com/) or Bill Bonner (http://dailyreckoning.com/author/bbonner/) to see how well that "powering by debt" is working out.

    ReplyDelete
    Replies
    1. Steve Keen is interesting. I have watched many of his presentations/lectures on Youtube and will be buying more of his books when I am where it is more practical.

      Keen is working on an interesting piece of software called Minsky (on Sourceforge) that does debt-based simulations of the economy.

      Delete
  3. But what's interesting here is that the tally sticks are used as essentially a carbon copy invoice for folks who can't read or write. The features of this system pretty directly give rise to double entry accounting. It seems to me there is room to look to internal controls for loosely coupled distributed transactions particularly those which involve money.

    For example, suppose you are transferring some money from bank A to bank B. The traditional way db folks are taught to think about this is through two phase commit but this isn't how this is done in the real world which is that several discrete data transfers occur. First the money in your account is "suspended." Then the bank sends information somehow (whether via a wire using TELEX or the like or ACH or something else) to the other bank. The other bank acknowledges that this is now payable to them, and they then put the money in your account over there. The money is then paid to the other bank against the funds suspended from your account.

    Not only is this guaranteed to be eventually consistent, this guarantee is possible because of the fact that it is done as a series of small, reversible steps, which are atomic in themselves and can be tied to other related steps on the other side. On other words instead of a single large transaction guaranteeing consistency on both sides, we have a series of loosely coupled transactions which are tracked, one side against the other. The answer is thus not to get rid of ACID compliance but to build other solutions on top of it.

    ReplyDelete
  4. But the eventual consistency depends on either an account reconciliation or a "settlement". In the case of wire transfers directly from bank A to bank B, someone (or some process) has to (later) receive the account statement from bank A and reconcile the entries in the statement to the ledger for that account at bank B (credits matched against debits and vice versa). In the case of ACH or similar clearing arrangements, the participants tally the inflows/outflows and have to verify the net positions in order for settlement to take place. If there are differences (particularly in the reconciliation), the eventual consistency can take a while to resolve itself.

    ReplyDelete
    Replies
    1. But the point is here is a case where the C in the CAP Theorem is not desirable. Humans are bound to the CAP theorem so tightly that little important work is done with current, absolute consistency. The major examples I can think of where consistency in the CAP sense is required involve entertainment (dancing, or playing a string quartet, and in these cases partition tolerance is 0-- if the first violinist can't continue playing on a string quartet neither can anyone else, and if your dancing partner sprains an ankle you are going to have to stop).

      Important work (like tracking the money in the petty cash drawer, or a inter-bank transfer) require agreement by parties prior to blessing of a consistent snapshot. Yes, you can see TPC as analogous to this but this requires seeing the agent issuing TPC calls as being one actor and not necessarily the only one. Again, if you and I run banks, we are not giving eachother access to eachothers' databases, so we have something very loosely coupled where people rather than machines are in control.

      Delete
    2. Chris,

      Banks do give (limited) access to each others databases. In the old days, if bank A had to make payments in French francs, they had an account at, say, BNP and would send telexes to BNP asking them to debit bank A's account for payment. BNP would then send an a bank statement to bank A and someone at bank A would reconcile each entry in the statement to bank A's accounts. Eventually SWIFT was created to enable banks to automate the processing of those transfers and since SWIFT messages had to have uniform identifiers, they also allowed the reconcilement function to be automated.

      The ACH is simply an extension of the check clearing procedures, whereby MICR-encoded checks were processed at the payor bank and then presented to the various draftee banks (effectively sharing information). Nowadays CHIPS, FedWire and ACH serve a similar purpose, in a highly automated fashion. People do get involved of course, particularly because if a bank's say CHIPS position is negative at end-of-day, they have to borrow Fed funds to cover, but that is mostly ancillary to the automated processes.

      Delete
    3. Joe: These are not the same thing as direct access to database. ACH, SWIFT, etc, can be thought of as being like a web service API. The API (like any payment API) has legal ramifications, but how it is handled internally is each bank's business.

      I can also tell you that none of these get posted to any financial institution's books without someone reviewing at least a summary of the information involved.

      It is hard to tell which is ancillary to what though. The human role is that of supervisor, and the computer is essentially given the role of clerk. The reasons are not just involving the position at the end of the day. It is also to guard against computer and/or human error.

      A telex is basically an electronic version of a paper voucher (and in fact in the old days would be printed out by a teleprinter). Everything beyond that is just a layer of automation on top of the paper voucher system. Humans get involved in a supervisory role and are incorporated into the system as active, CAP-theorem constrained components.

      Delete
    4. I have to disagree with somebody reviewing the postings. A CHIPS or FedWire payment gets received by a New York bank (I used to work for one of them), acknowledged automatically, and posted to the destination account without any human involvement on the receiving end. The only intervention is for the final settlement numbers. The payments are as automated as what happens when you swipe your debit card at a local store, key in your pin, and the merchant's bank and your bank settle the debit without further human intervention.

      Delete
    5. That's interesting. I do know my bank (a small bank) does a batch review of incoming and outgoing transactions. Maybe it varies from bank to bank.

      Delete
  5. Oh, and two-phase commit is in a sense analogous to the reconciliation/settlement. The coordinator acts like the ACH, asking each participant if their "tallies" are in sync, i.e., if they're ready to commit.

    ReplyDelete
  6. Hi! nice blog thank for sharing ..............
    Accounting Software Australia

    ReplyDelete
  7. Really the blogging is spreading its wings rapidly. Your write up is a fine example of this.It also cleared my doubt. Hats off to you!
    oracle ebs

    ReplyDelete
  8. Effectively managing your cash flow can help your business save money. Online accounting software makes it simple to determine incentives for employees and discounts for creditors or customers. In addition, the manager or business owner can view profits and losses in a comprehensible way.

    Station Accounting Software

    ReplyDelete