Historically there was very little cash outside of towns as everyone ran "tabs" until harvest time.
Only strangers were required to pay in cash and there were few of those away from main trade routes.

It seems to me that stealing tally sticks and debt records would have a completely different feel.

When "loot" is physical trade goods instead of coin, more pawn shops and fewer banks.

I don't have a problem with huge quantities of coin sloshing around a fantasy world, but I believe D&D has plenty of flexibility on this matter. The 4th Edition books (at least one DMG and the Dark Sun Campaign Guide) make mention of "alternate rewards" in terms of "favors" from useful or important people. Kill the dragon and maybe it has a few key items one can take, but the rest of the "gold" is more in the form of renown. For example: The paladin doesn't need to "buy" his next sword; his order just transfers a relic to him from their vault, as he has shown himself worthy of it. The town guard is willing to look the other way for the heroic thief, with the result that a magical cloak finds its way into his hands. Etc.

I suppose that's not exactly "credit." All that could be handled upfront, though. The party is provided with what they need for their exploits and they can keep it all if they do well. They can keep it all if they fail, but - assuming they survive - they find it harder to advance due to disgruntled lenders (i.e. they don't have enough experience yet).

Unless the game has a heavy focus on the PCs stealing physical things, or being robbed of their own gold, abstracting money should work just fine. I tend to prefer it, if my games make it to the point where the PCs are actually acquiring new stuff.

Well, credit is just a system of (theoretically) controlled debt.  So I would suggest that it would depend on how your society manages debt - and debtors.  Who keeps records of debt?  What form?  Tally sticks and records, to use your example, could be kept in a central location, with local businesses handing them over at the end of the day/week/whatever.  Perhaps that central repository is extremely secure and anyone wanting to avoid debt will need to attain such records before they reach the repository.

On a day-to-day basis I can imagine it would require either a certain level of trust between citizens (trust that the shopkeep giving you credit, for example, is recording it correctly), or a system of extremely accurate book-keeping with neutral auditors that deal out very harsh punishments for embezzlement (if trust can't be established and maintained).

However, if all of this is based around repayment of debt at harvest time, I can see an additional (and potentially interesting) complication: what produce has which value?  What happens if people go through the year on the assumption that potatoes will be worth ¤0.01 each, while pumpkins will be worth ¤0.50 each, but there's a rash of tater-rot while pumpkins flourish unusually, and suddenly with only one in each hundred spuds surviving to harvest, potatoes are far, far more in demand than pumpkins?  One agricultural disaster could completely change the distribution of wealth in the society - and there might be no warning until harvest time.  Mr and Mrs Potato can pay off their debts with ease even on a vastly reduced crop yield, whereas Messrs Pumpkin & Sons find themselves up to the eyeballs in debt they can't repay, as well as having a surplus of stock that will rot if it's not shifted soon.

Those complications are always possible, of course, but if the society is geared to use debt as its ongoing pre-harvest currency, it's a problem that will have a much greater impact if it arises.

DnD usually seems to run to itinerant heroes, which is a major problem for participating in a credit-based economy. Who will trust someone who is here today, gone tomorrow?

Although local credit for those with roots in the area is a viable model, keep in mind that the peasantry is going to be very short of material possessions, as well as ready cash. A typical farming/artisan family might have one set of homespun clothes each, a few homespun blankets/cloaks, a pot to cook porridge, a few kitchen implements, some bedframes (fill with straw), a table and some benches/stools, a cradle, some earthenware dishes/cups, a loom for weaving the aforementioned clothes/blankets, the tools of their trade, maybe a pig and some chickens, meagre stores of food.

Their lives (to paraphrase) will be nasty, brutish, ill-fed, and cold. They won't buy or sell anything  for coin, but transact everything by barter. The village/clan will hold in common plow animals, plows, etc.

The adventurers will be expected to pony up coin for their purchases or to barter. Even those dull, rusty goblin axe-heads may have a certain value to the smith who can reclaim the metal and maybe the handles can be traded to the innkeeper as firewood for a couple of ales.

In reply to Kioma (msg # 3):

In the past, most people have lived in one place all their lives and so credit can be based on trust.
Wandering adventurers on the other hand ... who do they know and who knows them?

Well, we now know why Messrs Pumpkin & Sons are willing to risk their lives in the dark depths of the earth, that nobody else would venture into.

Travelling merchants could make a living if the roads are safe and the local lord allowed.
Goods crafted in one village and sold in another, between approved "Market Days", would allow "Wandering Adventurers" to fit in.
Who's to say where the gear they have to exchange came from?