FIMK adds websocket support to it’s core

All crypto coin (distributed consensus) systems exists roughly of the same three building blocks.

You always have your p2p (peer to peer) part which sits in memory and talks to the other peers on the network, in FIMK this part is a java servlet running in a Jetty servlet container. Users don’t normally interact with this part.

Another part is the API (Application Programming Interface) which in the case of NXT, FIMK but also Bitcoin consists of an HTTP server (aka webserver) which accepts HTTP requests and returns responses in the form of JSON packets. The model here is that for each individual request to the API server (for instance give me block 100,000) you’ll get one individual response which contains the requested data.

The last part is probably the most well known part and is called the client. It’s the piece of software users interact with and install on their computers. The client allows you to login, watch your balance or send transactions. Examples of client software are Bitcoins QT wallet, the NXT NRS client and the FIMK client (currently known as mofowallet).

Bitcoin QT wallet

Bitcoin QT wallet (Language C++, framework QT)

NXT NRS client

NXT NRS client (Language Javascript)

FIMK client

FIMK client  (Language Javascript, framework Angular)

About the API server

To see with our own eyes how the API server works we can ask one of the public FIMK nodes to show us the details for block 100,000 (which I happen to know has id 11855029710905866888). To do this we first construct a URL in a way the the FIMK API server understands.

URL to ask API server for a block
https://fim6.mofowallet.org:7886/nxt?requestType=getBlock&block=11855029710905866888

If you click that URL you’ll get a response from the API server which runs at fim6.mofowallet.org. If we look at the contents of what is returned you’ll see a structure of key/value pairs (in this case in JSON notation) where the keys remain the same for each response only the values will differ based on the arguments passed to the API server. This basically is your Interface in API.

{
  "transactions":[],
  "nextBlock":"512348776372078773",
  "generatorRS":"FIM-6JXS-8RYY-8H8K-52ACJ",
  "totalAmountNQT":"0",
  "blockSignature":"c9fab61e98873afd408f1a99ed20f9c13778da9b29c3032121050b8ffd57210aa5c5888c814b3a01f2bd6e1eb36354118a025421b47a1d7d216bf623c12c884c",
  "payloadLength":0,
  "generatorPublicKey":"c6479b05732e73d0ca41eb779e2779f77f63e180b68fb2e84cdd0e591b70eb2a",
  "numberOfTransactions":0,
  "version":3,
  "timestamp":24286135,
  "previousBlock":"16798496361687630053",
  "payloadHash":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "height":100000,
  "totalFeeNQT":"0",
  "baseTarget":"5831850624",
  "generationSignature":"05f15452b3d5da0787ee0776ddb5c8a7383d35ca877e98bc1ed4d221735be6e2",
  "previousBlockHash":"e540154f703f20e9bc86086e859e09d5972ee36dbe2f52adb42d4011a99695a0",
  "generator":"4046765972061832120",
  "cumulativeDifficulty":"1015633248529448",
  "totalPOSRewardNQT":"20000000000"
}

As you see the API server in NXT, FIMK or Bitcoin allows external access to it’s data and this is exactly how all crypto coin clients gather the information you see when you use one of those clients (actually the FIMK client used to work like that but more on that later).

Push vs pull model

The API servers in FIMK, NXT and Bitcoin operate on a so called pull model. The client in this case pulls in the data from the server, in this model the server has no way of pushing anything to the client, only when the client asks will the server answer. Push communication works differently, here the client opens a connection to server that is two way. With push the server can send something to the client without the client having to ask for it, this is what enables the real time web.

Push vs pull

Push vs pull

But what’s wrong with the pull model? You could argue it has worked very good for both NXT and Bitcoin QT clients and in all honesty both NXT and Bitcoin clients seem to be showing the latest data from the peer 2 peer network near instantly. This however is not what it seems, what actually happens is a timer in the client will ask the server if there is anything new to report which it does not do every second or even every 10 seconds, to not stress the network too much these updates only happen as much as once every 30 seconds (NXT NRS client) how long these dark periods last in Bitcoin QT is unclear to me (enlighten me in the comments if you know the answer).

Earlier versions of mofowallet worked in the same way as the current NXT and Bitcoin QT clients during those times I’ve continuously struggled to try and bring the dark period between those intervals down to an acceptable level. We tried 1 second or 5 second intervals but always ran into limitations involving both the client (slowing down the computer it runs on) and the server not being able to keep up with the constant repeating requests.

The p2p network is real time/instant

Now we know that crypto coin clients only update once every 30 seconds (not sure about Bitcoin QT) we should also be aware that everything that happens on the server side happens instantly and in real time. The moment you hit the send button to broadcast a transaction the server you send that transaction to will instantly send that transaction to all other connected peers (other servers) this happens in a fraction of a second.

Now a weird situation has been created. A transaction was send to the network and accepted by all peers and in all likelihood that transaction will become part of the blockchain, yet none of the connected clients is aware of this. In the worst case those clients are not aware for the coming 30 seconds.

This in-balance between what the client knows and what the server knows is not a big problem for first generation coins like Bitcoin, all you can basically do with the QT client is send Bitcoin to someone else. However for second generation coins like NXT and FIMK this can be a problem and especially when it comes to the built-in decentralized Asset Exchange or functionalities such as trading Monetary System coins. Not being aware of accepted transactions (think buy or sell orders) is not the best position to be in when trading assets on the Asset Exchange.

The customer wants real time/instant

So the million dollar question is how to get rid of the 30 second dark periods between intervals. How to make the client just as real time and instant as the peer 2 peer network it says it is showing you? When making decisions, to buy an asset at a certain price, you should be as much in the know as possible. The fact that someone else placed a sell order should be known instantly and not after 30 seconds. It could cost you money not knowing that.

Another plus for instant when compared to 30 second intervals is the speed at which you can do business. All the big (and small) bitcoin payment processors now operate over instant communication channels (called websockets but more on that later) it’s what allows you to make a bitcoin payment from your wallet at blockchain.info and see it apear almost instantly on the checkout page when ordering that pizza, no need to wait for a block before your pizza is put in the oven.

getting hungry

mmmm pizza

The way these payment processors do this is by running their bitcoin daemon which they configured to send a signal to a program or server when a transaction arrives over the peer 2 peer network. That real time signal they then get is sent instantly over the already opened communication channel they have with your browser when you are on the checkout page ordering your pizza.

FIMK adds real time/instant to it’s core

Since FIMK is a second generation crypto currency (unlike Bitcoin) and comes with new functionalities such as the Asset Exchange and these functionalities require instant/real time clients. Where Bitcoin can do without real time clients, FIMK simply can’t and action on our parts was required.

It is (partially) because of this that we stopped work on our own version of the forked NXT NRS client and figured that the way to offer our users that real time experience was to build our own client from the ground up. Not relying on the HTTP API but on Websockets exclusively. We stopped to rely on the available API server functions (in which NXT really is in a league of it’s own when it comes to the number of API calls available) and instead started developing the client in the first place and then expose the required server hooks that the client requires (this instead of building our client around the existing API).

Just as the “standard” API server in NXT and other NXT clones runs on it’s own port we have added a new server to the core which also listens on it’s own port. In many ways this server is similar to the existing API server, it can be configured in the same ways the existing API server can (it even shares some configuration settings). It also can be completely turned off as you can turn off the API server. Finally there is a bridge available to call each of the existing API server calls from the Websocket server without the need to even start the API server (so no functionality is lost when you switch to the new server).

And of course we have added SSL (HTTPS) support to our Websocket server which allows clients to connect over the secured wss:// protocol.

3rd party connectivity

Right now only the FIMK client makes use of the real time capabilities of the FIMK server but since WebSockets are an open standard you can connect to FIMK server from basically any programming language. Just to name a few..

  1. C#
  2. NodeJS
  3. Silverlight
  4. .NET, alternative
  5. Java, or this one
  6. C++ (arduino)
  7. Ruby, or event machine based
  8. Python, or even better
  9. Objective-C, or this one
  10. And of course Firefox, Chrome, Opera and Internet Explorer through native Javascript.

Basically any modern language used today has a websocket client library and can connect to the FIMK Websocket server. Merchants can offer full checkout support using nothing but a running FIMK server and some Javascript, the client will see instant confirmations right in his browser. Block Explorers can connect to a FIMK server and show real time data about new blocks, unconfirmed transactions or anything else.

Websocket support status

The websocket server in FIMK core is considered ready for use and has been running on a selection of association public nodes for a couple of weeks now. The integration of the FIMK client (mofowallet) with the Websocket server is mostly done. What remains are various providers (more on that in a later post) that are able to get their data from the Websocket server but have not yet implemented the various callbacks a FIMK Websocket client can expect. Not much work but still.

If you want to take Websocket enabled mofowallet for a spin you can do that at our hosted version.

Thank you for reading this far.

Share Your Thoughts

Leave a Reply