Developing a digital currency using the Crystal programming language

This article aims to demystify the blockchain by delving into its inner workings. While the original bitcoin whitepaper provided a starting point, I believe that practical experience is crucial for true comprehension.

Thus, I embarked on a journey to create CrystalCoin, a cryptocurrency built from the ground up using the Crystal programming language. This article won’t delve into the intricacies of algorithm selection, hash difficulty, or related concepts. Instead, it will provide a tangible example to illustrate the strengths and limitations of blockchains.

For those seeking a primer on algorithms and hashing, I recommend Demir Selmanovic’s article “Cryptocurrency for Dummies: Bitcoin and Beyond.”

Why Crystal?

I sought a language that combined the productivity of Ruby with robust performance. Cryptocurrencies demand significant computational power for tasks like mining and hashing, making compiled languages like C++ and Java popular choices. However, I wanted a language with a more elegant syntax to maintain an enjoyable development process and enhance code readability. Crystal’s performance is already quite impressive.

Crystal Programming Language illustration

So, why did I opt for the Crystal programming language? Crystal’s syntax closely resembles Ruby’s, making it intuitive to grasp and write code in. This similarity translates into a gentler learning curve, particularly for seasoned Ruby developers.

The Crystal language team aptly describes it as:

Fast as C, slick as Ruby.

Unlike interpreted languages like Ruby or JavaScript, Crystal is compiled, resulting in faster execution and a reduced memory footprint. It leverages LLVM to compile code into native instructions.

Furthermore, Crystal’s static typing allows the compiler to detect type errors during compilation.

While Crystal’s merits deserve an article of their own, I encourage you to explore this article for a comprehensive overview of its capabilities.

Note: This article assumes a basic understanding of Object Oriented Programming (OOP).

Deconstructing the Blockchain

A blockchain is essentially an immutable ledger—a chain of blocks—safeguarded by cryptographic hashes, which act like digital fingerprints.

Imagine it as a linked list where each element not only points to the previous one but also incorporates its hash into its own identifier. This interdependence ensures that modifying a block necessitates recomputing all subsequent blocks.

Think of a blockchain as a series of data-containing blocks linked by a chain of hashes, where each hash depends on the previous block’s content.

This blockchain is not confined to a single server; instead, it resides on every node participating in the network, making it decentralized. Each node maintains a complete copy of the blockchain (over 149 GB for Bitcoin as of December 2017).

Hashing and Digital Signatures: The Guardians of Integrity

A hash function generates a unique fingerprint for any given input, whether it’s text or an object. Even the slightest alteration to the input drastically changes the resulting fingerprint.

This article employs the SHA256 hashing algorithm, the same one used by Bitcoin.

SHA256 consistently produces a 64-character (256-bit) hexadecimal hash, regardless of the input’s size:

Input	Hashed Results
VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEX VERY LONG TEXT VERY LONG VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT	cf49bbb21c8b7c078165919d7e57c145ccb7f398e7b58d9a3729de368d86294a
Toptal	2e4e500e20f1358224c08c7fb7d3e0e9a5e4ab7a013bfd6774dfa54d7684dd21
Toptal.	12075307ce09a6859601ce9d451d385053be80238ea127c5df6e6611eed7c6f0

Notice how adding a simple . (dot) in the last example significantly alters the hash.

In a blockchain, each block’s data is fed into the hashing algorithm to generate a unique hash that links it to the next block, creating a chain of blocks secured by their predecessors’ hashes.

Building CrystalCoin: A Practical Implementation

Let’s initiate our Crystal project and set up the SHA256 encryption mechanism.

Assuming you have Crystal programming language installed installed, we’ll use Crystal’s built-in project scaffolding tool, crystal init app [name], to create the project’s foundation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
% crystal init app crystal_coin
      create  crystal_coin/.gitignore
      create  crystal_coin/.editorconfig
      create  crystal_coin/LICENSE
      create  crystal_coin/README.md
      create  crystal_coin/.travis.yml
      create  crystal_coin/shard.yml
      create  crystal_coin/src/crystal_coin.cr
      create  crystal_coin/src/crystal_coin/version.cr
      create  crystal_coin/spec/spec_helper.cr
      create  crystal_coin/spec/crystal_coin_spec.cr
Initialized empty Git repository in /Users/eki/code/crystal_coin/.git/

This command generates the basic project structure, including an initialized Git repository, license, and readme files. It also provides stubs for tests and a shard.yml file for project description and dependency management (shards).

Next, we’ll incorporate the openssl shard, essential for the SHA256 algorithm:

1
2
3
4
# shard.yml
dependencies:
  openssl:
    github: datanoise/openssl.cr

Running crystal deps in your terminal will download openssl and its dependencies.

With the necessary library in place, let’s define our Block class and construct the hash function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
# src/crystal_coin/block.cr

require "openssl"

module CrystalCoin
  class Block

    def initialize(data : String)
      @data = data
    end

    def hash
      hash = OpenSSL::Digest.new("SHA256")
      hash.update(@data)
      hash.hexdigest
    end
  end
end

puts CrystalCoin::Block.new("Hello, Cryptos!").hash

You can test your code by executing crystal run crystal src/crystal_coin/block.cr.

1
2
crystal_coin [master●] % crystal src/crystal_coin/block.cr
33eedea60b0662c66c289ceba71863a864cf84b00e10002ca1069bf58f9362d5

Crafting the Blockchain Structure

Each block stores a timestamp, an optional index, and a self-identifying hash for integrity. Mimicking Bitcoin, each block’s hash in CrystalCoin is a cryptographic hash of its index, timestamp, data, and the previous block’s hash (previous_hash). The data field can hold arbitrary information for now.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
module CrystalCoin
  class Block

    property current_hash : String

    def initialize(index = 0, data = "data", previous_hash = "hash")
      @data = data
      @index = index
      @timestamp = Time.now
      @previous_hash = previous_hash
      @current_hash = hash_block
    end

    private def hash_block
      hash = OpenSSL::Digest.new("SHA256")
      hash.update("#{@index}#{@timestamp}#{@data}#{@previous_hash}")
      hash.hexdigest
    end
  end
end


puts CrystalCoin::Block.new(data: "Same Data").current_hash

Crystal replaces Ruby’s attr_accessor, attr_getter, and attr_setter methods with dedicated keywords:

Ruby Keyword	Crystal Keyword
attr_accessor	property
attr_reader	getter
attr_writer	setter

Additionally, Crystal encourages type hinting to guide the compiler. While type inference is present, explicit declarations improve clarity. That’s why we specified String for current_hash.

Running block.cr twice with the same data yields different hashes due to varying timestamp values:

1
2
3
4
crystal_coin [master●] % crystal src/crystal_coin/block.cr
361d0df74e28d37b71f6c5f579ee182dd3d41f73f174dc88c9f2536172d3bb66
crystal_coin [master●] % crystal src/crystal_coin/block.cr
b1fafd81ba13fc21598fb083d9429d1b8a7e9a7120dbdacc7e461791b96b9bf3

Now that we have our block structure, we need to link them into a blockchain. Each block relies on information from its predecessor, but how does the first block, the genesis block, come into existence? The genesis block is unique—it has no predecessors and is often added manually or through specific logic.

Let’s create a Block.first class method to generate the genesis block with index=0, arbitrary data, and an arbitrary previous_hash:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
module CrystalCoin
  class Block
    ...
    
    def self.first(data="Genesis Block")
      Block.new(data: data, previous_hash: "0")
    end
    
    ...
  end
end

Testing this with p CrystalCoin::Block.first yields:

1
#<CrystalCoin::Block:0x10b33ac80 @current_hash="acb701a9b70cff5a0617d654e6b8a7155a8c712910d34df692db92455964d54e", @data="Genesis Block", @index=0, @timestamp=2018-05-13 17:54:02 +03:00, @previous_hash="0">

With the genesis block in place, we need a function to generate subsequent blocks. This function takes the previous block as input, creates the new block’s data, and returns the new block with the appropriate hash. By incorporating information from previous blocks, each new block strengthens the blockchain’s integrity.

This interdependence ensures that altering a block requires modifying all following blocks’ hashes. For example, changing the data in block 44 from LOOP to EAST necessitates changing all subsequent hashes because each hash depends on the previous_hash value.

Without this mechanism, malicious actors could easily tamper with the data and replace the chain. This chain of hashes serves as cryptographic proof, ensuring the immutability of the blockchain.

Let’s implement the Block.next class method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
module CrystalCoin
  class Block
    ...
    
    def self.next(previous_node, data = "Transaction Data")
      Block.new(
        data: "Transaction data number (#{previous_node.index + 1})",
        index: previous_node.index + 1,
        previous_hash: previous_hash.hash
      )
    end
    ...
  end
end   

To illustrate, we’ll create a simple blockchain starting with the genesis block and add five succeeding blocks:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
blockchain = [ CrystalCoin::Block.first ]

previous_block = blockchain[0]

5.times do

  new_block  = CrystalCoin::Block.next(previous_block: previous_block)

  blockchain << new_block

  previous_block = new_block

end

p blockchain

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
[#<CrystalCoin::Block:0x108c57c80

  @current_hash=

   "df7f9d47bee95c9158e3043ddd17204e97ccd6e8958e4e41dacc7f0c6c0df610",

  @index=0,

  @previous_hash="0",

  @timestamp=2018-06-04 12:13:21 +03:00,

  @data="Genesis Block>,

 #<CrystalCoin::Block:0x109c89740

  @current_hash=

   "d188fcddd056c044c783d558fd6904ceeb9b2366339af491a293d369de4a81f6",

  @index=1,

  @previous_hash=

   "df7f9d47bee95c9158e3043ddd17204e97ccd6e8958e4e41dacc7f0c6c0df610",

  @timestamp=2018-06-04 12:13:21 +03:00,

  @data="Transaction data number (1)">,

 #<CrystalCoin::Block:0x109cc8500

  @current_hash=

   "0b71b61118b9300b4fe8cdf4a7cbcc1dac4da7a8a3150aa97630994805354107",

  @index=2,

  @previous_hash=

   "d188fcddd056c044c783d558fd6904ceeb9b2366339af491a293d369de4a81f6",

  @timestamp=2018-06-04 12:13:21 +03:00,

  @transactions="Transaction data number (2)">,

 #<CrystalCoin::Block:0x109ccbe40

  @current_hash=

   "9111406deea4f07f807891405078a3f8537416b31ab03d78bda3f86d9ae4c584",

  @index=3,

  @previous_hash=

   "0b71b61118b9300b4fe8cdf4a7cbcc1dac4da7a8a3150aa97630994805354107",

  @timestamp=2018-06-04 12:13:21 +03:00,

  @transactions="Transaction data number (3)">,

 #<CrystalCoin::Block:0x109cd0980

  @current_hash=

   "0063bfc5695c0d49b291a8813c566b047323f1808a428e0eb1fca5c399547875",

  @index=4,

  @previous_hash=

   "9111406deea4f07f807891405078a3f8537416b31ab03d78bda3f86d9ae4c584",

  @timestamp=2018-06-04 12:13:21 +03:00,

  @transactions="Transaction data number (4)">,

 #<CrystalCoin::Block:0x109cd0100

  @current_hash=

   "00a0c70e5412edd7389a0360b48c407ce4ddc8f14a0bcf16df277daf3c1a00c7",

  @index=5,

  @previous_hash=

   "0063bfc5695c0d49b291a8813c566b047323f1808a428e0eb1fca5c399547875",

  @timestamp=2018-06-04 12:13:21 +03:00,

  @transactions="Transaction data number (5)">

Proof-of-Work: Securing the Network

A Proof of Work (PoW) algorithm governs the creation, or mining, of new blocks. Its purpose is to find a number that solves a computational problem—a number that is computationally expensive to find but easy to verify.

Let’s illustrate with a simplified example. Suppose a valid hash requires the hash of the product of integers x and y to start with 00:

1
hash(x * y) = 00ac23dc...

Let’s fix x to 5 and implement this in Crystal:

1
2
3
4
5
6
7
8
9
x = 5
y = 0

while hash((x*y).to_s)[0..1] != "00"
  y += 1
end

puts "The solution is y = #{y}"
puts "Hash(#{x}*#{y}) = #{hash((x*y).to_s)}"

Running the code gives us:

1
2
3
4
crystal_coin [master●●] % time crystal src/crystal_coin/pow.cr
The solution is y = 530
Hash(5*530) = 00150bc11aeeaa3cdbdc1e27085b0f6c584c27e05f255e303898dcd12426f110
crystal src/crystal_coin/pow.cr  1.53s user 0.23s system 160% cpu 1.092 total

Finding the number y=530 required brute-force computation, but verifying it is simple using the hash function.

PoW is crucial for blockchain security. In our example, a valid hash starts with 00. This criterion is the difficulty; higher difficulty increases the time to find a valid hash.

To find a valid hash, we introduce a nonce—a number incremented with each attempt. We combine the block data (date, message, previous hash, index) with a nonce, starting from 1. If the resulting hash is not valid, we increment the nonce and try again until we find a valid hash.

Bitcoin’s PoW algorithm is called Hashcash. Let’s add a proof-of-work mechanism to our Block class with a hardcoded difficulty of two leading zeros (00):

Our updated CrystalCoin Block will have these attributes:

1
2
3
4
5
6
1) index: indicates the index of the block ex: 0,1
2) timestamp: timestamp in epoch, number of seconds since 1 Jan 1970
3) data: the actual data that needs to be stored on the blockchain.
4) previous_hash: the hash of the previous block, this is the chain/link between the blocks
5) nonce: this is the number that is to be mined/found.
6) current_hash: The hash value of the current block, this is generated by combining all the above attributes and passing it to a hashing algorithm

For modularity, we’ll create a separate proof_of_work.cr module:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
require "openssl"

module CrystalCoin
  module ProofOfWork

    private def proof_of_work(difficulty = "00")
      nonce = 0
      loop do
        hash = calc_hash_with_nonce(nonce)
        if hash[0..1] == difficulty
          return nonce
        else
          nonce += 1
        end
      end
    end

    private def calc_hash_with_nonce(nonce = 0)
      sha = OpenSSL::Digest.new("SHA256")
      sha.update("#{nonce}#{@index}#{@timestamp}#{@data}#{@previous_hash}")
      sha.hexdigest
    end
  end
end

Our modified Block class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
require "./proof_of_work"

module CrystalCoin
  class Block
    include ProofOfWork

    property current_hash : String
    property index : Int32
    property nonce : Int32
    property previous_hash : String


    def initialize(index = 0, data = "data", previous_hash = "hash")
      @data = data
      @index = index
      @timestamp = Time.now
      @previous_hash = previous_hash
      @nonce = proof_of_work
      @current_hash = calc_hash_with_nonce(@nonce)
    end

    def self.first(data = "Genesis Block")
      Block.new(data: data, previous_hash: "0")
    end

    def self.next(previous_block, data = "Transaction Data")
      Block.new(
        data: "Transaction data number (#{previous_block.index + 1})",
        index: previous_block.index + 1,
        previous_hash: previous_block.current_hash
      )
    end
  end
end

It’s worth noting some Crystal-specific aspects. Unlike Ruby, where methods are public by default, Crystal requires explicit private keywords for private methods.

Crystal offers a wider range of integer types (Int8, Int16, Int32, Int64, UInt8, UInt16, UInt32, UInt64) compared to Ruby’s Fixnum. Similarly, true and false belong to the Bool class, unlike Ruby’s TrueClass and FalseClass.

Crystal natively supports optional and named method arguments, simplifying argument handling. For instance, Block#initialize(index = 0, data = "data", previous_hash = "hash") can be called with Block.new(data: data, previous_hash: "0").

For a deeper dive into the differences between Crystal and Ruby, refer to Crystal for Rubyists.

Now, let’s create five transactions using:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
blockchain = [ CrystalCoin::Block.first ]
puts blockchain.inspect
previous_block = blockchain[0]

5.times do |i|
  new_block  = CrystalCoin::Block.next(previous_block: previous_block)
  blockchain << new_block
  previous_block = new_block
  puts new_block.inspect
end

1
2
3
4
5
6
[#<CrystalCoin::Block:0x108f8fea0 @current_hash="0088ca080a49334e1cb037ed4c42795d635515ef1742e6bcf439bf0f95711759", @index=0, @nonce=17, @timestamp=2018-05-14 17:20:46 +03:00, @data="Genesis Block", @previous_hash="0">]
#<CrystalCoin::Block:0x108f8f660 @current_hash="001bc2b04d7ad8ef25ada30e2bde19d7bbbbb3ad42348017036b0d4974d0ccb0", @index=1, @nonce=24, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (1)", @previous_hash="0088ca080a49334e1cb037ed4c42795d635515ef1742e6bcf439bf0f95711759">
#<CrystalCoin::Block:0x109fc5ba0 @current_hash="0019256c998028111838b872a437cd8adced53f5e0f8f43388a1dc4654844fe5", @index=2, @nonce=61, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (2)", @previous_hash="001bc2b04d7ad8ef25ada30e2bde19d7bbbbb3ad42348017036b0d4974d0ccb0">
#<CrystalCoin::Block:0x109fdc300 @current_hash="0080a30d0da33426a1d4f36d870d9eb709eaefb0fca62cc68e497169c5368b97", @index=3, @nonce=149, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (3)", @previous_hash="0019256c998028111838b872a437cd8adced53f5e0f8f43388a1dc4654844fe5">
#<CrystalCoin::Block:0x109ff58a0 @current_hash="00074399d51c700940e556673580a366a37dec16671430141f6013f04283a484", @index=4, @nonce=570, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (4)", @previous_hash="0080a30d0da33426a1d4f36d870d9eb709eaefb0fca62cc68e497169c5368b97">
#<CrystalCoin::Block:0x109fde120 @current_hash="00720bb6e562a25c19ecd2b277925057626edab8981ff08eb13773f9bb1cb842", @index=5, @nonce=475, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (5)", @previous_hash="00074399d51c700940e556673580a366a37dec16671430141f6013f04283a484">

Notice the difference? All hashes now begin with 00, demonstrating the effectiveness of proof-of-work. Our ProofOfWork module found the appropriate nonce to generate hashes meeting the difficulty requirement.

The first block required 17 nonce attempts:

Block	Loops / Number of Hash Calculations
#0	17
#1	24
#2	61
#3	149
#4	570
#5	475

Increasing the difficulty to four leading zeros (difficulty="0000") increases the computational effort:

Block	Loops / Number of Hash Calculations
#1	26 762
#2	68 419
#3	23 416
#4	15 353

The first block now required 26762 nonce attempts (compared to 17 with a difficulty of 00).

Exposing the Blockchain as an API

We’ve successfully implemented a basic blockchain, but it’s currently limited to a single machine. Let’s introduce decentralization by enabling our blockchain to run on multiple nodes.

From this point onward, CrystalCoin will use JSON for data exchange. Each block’s data field (now referred to as transactions) will contain a list of transactions in JSON format.

Each transaction object will detail the sender, receiver, and the amount of CrystalCoin being transferred:

1
2
3
4
5
{
  "from": "71238uqirbfh894-random-public-key-a-alkjdflakjfewn204ij",
  "to": "93j4ivnqiopvh43-random-public-key-b-qjrgvnoeirbnferinfo",
  "amount": 3
}

Let’s modify our Block class to accommodate the new transaction format:

We introduce a simple Transaction class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
module CrystalCoin
  class Block
    class Transaction

      property from : String
      property to : String
      property amount : Int32

      def initialize(@from, @to, @amount)
      end
    end
  end
end

Transactions are grouped into blocks, with each block potentially holding multiple transactions. These blocks are added to the blockchain at regular intervals.

To manage the collection of blocks, we introduce the Blockchain class:

Blockchain maintains two arrays: chain for mined blocks and uncommitted_transactions for pending transactions yet to be included in the blockchain. Upon initialization, Blockchain creates the genesis block using Block.first and adds it to the chain array. It also initializes an empty uncommitted_transactions array.

We’ll add the Blockchain#add_transaction method to append transactions to the uncommitted_transactions array.

Let’s build the Blockchain class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
require "./block"
require "./transaction"

module CrystalCoin
  class Blockchain
    getter chain
    getter uncommitted_transactions

    def initialize
      @chain = [ Block.first ]
      @uncommitted_transactions = [] of Block::Transaction
    end

    def add_transaction(transaction)
      @uncommitted_transactions << transaction
    end
  end
end

We’ll update the Block class to use transactions instead of data:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
module CrystalCoin
  class Block
    include ProofOfWork

    def initialize(index = 0, transactions = [] of Transaction, previous_hash = "hash")
      @transactions = transactions
      ...
    end

    ....

    def self.next(previous_block, transactions = [] of Transaction)
      Block.new(
        transactions: transactions,
        index: previous_block.index + 1,
        previous_hash: previous_block.current_hash
      )
    end

  end
end

To facilitate adding transactions to our blockchain network, we’ll create a simple HTTP server representing a node.

Our server will expose four endpoints:

[POST] /transactions/new: Creates a new transaction and adds it to the uncommitted transaction pool.
[GET] /mine: Instructs the server to mine a new block.
[GET] /chain: Returns the entire blockchain in JSON format.
[GET] /pending: Returns pending transactions (uncommitted_transactions).

We’ll utilize the Kemal web framework, a lightweight framework inspired by Ruby’s Sinatra. It simplifies mapping endpoints to Crystal functions. For those familiar with Ruby on Rails, Amber provides a comparable experience.

Let’s add Kemal to our shard.yml file and install it:

1
2
3
dependencies:
  kemal:
    github: kemalcr/kemal

Now, let’s construct the basic HTTP server:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# src/server.cr

require "kemal"
require "./crystal_coin"

# Generate a globally unique address for this node
node_identifier = UUID.random.to_s

# Create our Blockchain
blockchain = Blockchain.new

get "/chain" do
  "Send the blockchain as json objects"
end

get "/mine" do
  "We'll mine a new Block"
end

get "/pending" do
  "Send pending transactions as json objects"
end

post "/transactions/new" do
  "We'll add a new transaction"
end

Kemal.run

Let’s start the server:

1
2
crystal_coin [master●●] % crystal run src/server.cr
[development] Kemal is ready to lead at http://0.0.0.0:3000

Verifying the server’s functionality:

1
2
% curl http://0.0.0.0:3000/chain
Send the blockchain as json objects%

With the server running smoothly, let’s implement the endpoints. We’ll begin with /transactions/new and /pending:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
get "/pending" do
  { transactions: blockchain.uncommitted_transactions }.to_json
end

post "/transactions/new" do |env|

  transaction = CrystalCoin::Block::Transaction.new(
    from: env.params.json["from"].as(String),
    to:  env.params.json["to"].as(String),
    amount:  env.params.json["amount"].as(Int64)

  )

  blockchain.add_transaction(transaction)

  "Transaction #{transaction} has been added to the node"
end

These implementations are straightforward. We create a CrystalCoin::Block::Transaction object and add it to the uncommitted_transactions array using Blockchain#add_transaction.

Currently, transactions reside in the uncommitted_transactions pool until they are mined into a block. Mining involves selecting transactions from this pool, creating a new block using the Block.next method, and finding a valid nonce that satisfies the PoW criteria.

In CrystalCoin, we’ll use our previously defined Proof-of-Work algorithm. Mining a new block involves:

Identifying the last block in the chain.
Retrieving pending transactions (uncommitted_transactions).
Constructing a new block using Block.next.
Appending the mined block to the chain array.
Clearing the uncommitted_transactions pool.

Let’s implement the Blockchain#mine method to encapsulate these steps:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
module CrystalCoin
  class Blockchain
    include Consensus

    BLOCK_SIZE = 25

    ...
    
    def mine
       raise "No transactions to be mined" if @uncommitted_transactions.empty?

       new_block = Block.next(
         previous_block: @chain.last,
         transactions: @uncommitted_transactions.shift(BLOCK_SIZE)
       )

       @chain << new_block
    end
  end
end

We first check for pending transactions. Then, we retrieve the last block and select the first 25 transactions from the uncommitted_transactions pool using Array#shift(BLOCK_SIZE).

Now, let’s implement the /mine endpoint:

1
2
3
4
get "/mine" do
  blockchain.mine
  "Block with index=#{blockchain.chain.last.index} is mined."
end

And the /chain endpoint:

1
2
3
get "/chain" do
  { chain: blockchain.chain }.to_json
end

Interacting with the Blockchain

We’ll use cURL to interact with our API over the network.

First, start the server:

1
2
crystal_coin [master] % crystal run src/server.cr
[development] Kemal is ready to lead at http://0.0.0.0:3000

Next, create two transactions by sending POST requests to http://localhost:3000/transactions/new with the transaction data:

1
2
3
4
crystal_coin [master●] % curl -X POST http://0.0.0.0:3000/transactions/new -H "Content-Type: application/json" -d '{"from": "eki", "to":"iron_man", "amount": 1000}'
Transaction #<CrystalCoin::Block::Transaction:0x10c4159f0 @from="eki", @to="iron_man", @amount=1000_i64> has been added to the node%                                               
crystal_coin [master●] % curl -X POST http://0.0.0.0:3000/transactions/new -H "Content-Type: application/json" -d '{"from": "eki", "to":"hulk", "amount": 700}'
Transaction #<CrystalCoin::Block::Transaction:0x10c415810 @from="eki", @to="hulk", @amount=700_i64> has been added to the node%

Let’s list the pending transactions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
crystal_coin [master●] % curl http://0.0.0.0:3000/pendings
{
  "transactions":[
    {
      "from":"ekis",
      "to":"huslks",
      "amount":7090
    },
    {
      "from":"ekis",
      "to":"huslks",
      "amount":70900
    }
  ]
}

As expected, our two transactions are in the uncommitted_transactions pool.

Now, let’s mine these transactions by sending a GET request to http://0.0.0.0:3000/mine:

1
2
crystal_coin [master●] % curl http://0.0.0.0:3000/mine
Block with index=1 is mined.

We’ve successfully mined the first block and added it to our chain. Let’s verify:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
crystal_coin [master●] % curl http://0.0.0.0:3000/chain
{
  "chain": [
    {
      "index": 0,
      "current_hash": "00d469d383005b4303cfa7321c02478ce76182564af5d16e1a10d87e31e2cb30",
      "nonce": 363,
      "previous_hash": "0",
      "transactions": [
        
      ],
      "timestamp": "2018-05-23T01:59:52+0300"
    },
    {
      "index": 1,
      "current_hash": "003c05da32d3672670ba1e25ecb067b5bc407e1d5f8733b5e33d1039de1a9bf1",
      "nonce": 320,
      "previous_hash": "00d469d383005b4303cfa7321c02478ce76182564af5d16e1a10d87e31e2cb30",
      "transactions": [
        {
          "from": "ekis",
          "to": "huslks",
          "amount": 7090
        },
        {
          "from": "ekis",
          "to": "huslks",
          "amount": 70900
        }
      ],
      "timestamp": "2018-05-23T02:02:38+0300"
    }
  ]
}

Consensus and Decentralization: The Heart of the Blockchain

Our blockchain accepts transactions and mines new blocks, but it’s still confined to a single machine. The essence of a blockchain lies in its decentralization. But how do we maintain consistency across multiple nodes?

This is where the concept of Consensus comes into play.

We need a consensus algorithm to manage multiple nodes and ensure they all agree on the same blockchain state.

Registering New Nodes

To implement consensus, nodes need to be aware of their peers. Each node maintains a registry of other nodes in the network.

Let’s add two more endpoints:

[POST] /nodes/register: Accepts a list of new node URLs for registration.
[GET] /nodes/resolve: Implements our Consensus Algorithm to resolve conflicts and ensure chain consistency.

We’ll modify our blockchain’s constructor and introduce a node registration mechanism:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
--- a/src/crystal_coin/blockchain.cr
+++ b/src/crystal_coin/blockchain.cr
@@ -7,10 +7,12 @@ module CrystalCoin

     getter chain
     getter uncommitted_transactions
+    getter nodes

     def initialize
       @chain = [ Block.first ]
       @uncommitted_transactions = [] of Block::Transaction
+      @nodes = Set(String).new [] of String
     end

     def add_transaction(transaction)

We use a Set data structure to store node URLs, ensuring idempotency—each node appears only once, regardless of how many times it’s added.

Let’s create a new Consensus module and implement the register_node(address) method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
require "uri"

module CrystalCoin
  module Consensus
    def register_node(address : String)
      uri = URI.parse(address)
      node_address = "#{uri.scheme}:://#{uri.host}"
      node_address = "#{node_address}:#{uri.port}" unless uri.port.nil?
      @nodes.add(node_address)
    rescue
      raise "Invalid URL"
    end
  end
end

The register_node function parses and formats the node URL.

Now, let’s implement the /nodes/register endpoint:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
post "/nodes/register" do |env|
  nodes = env.params.json["nodes"].as(Array)

  raise "Empty array" if nodes.empty?

  nodes.each do |node|
    blockchain.register_node(node.to_s)
  end

  "New nodes have been added: #{blockchain.nodes}"
end

In a decentralized network, chain discrepancies might arise. Our consensus algorithm must resolve these conflicts to maintain system integrity. We’ll adopt the “longest valid chain” rule—the longest valid chain becomes the authoritative version. This approach assumes that the longest chain represents the most computational effort invested.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
module CrystalCoin
  module Consensus
    ...
    
    def resolve
      updated = false

      @nodes.each do |node|
        node_chain = parse_chain(node)
        return unless node_chain.size > @chain.size
        return unless valid_chain?(node_chain)
        @chain = node_chain
        updated = true
      rescue IO::Timeout
        puts "Timeout!"
      end

      updated
    end
    
  ...
  end
end

The resolve method iterates through neighboring nodes, downloads their chains, and verifies them using the valid_chain? method. If a longer valid chain is found, it replaces the current node’s chain.

Now, let’s implement the private methods parse_chain() and valid_chain?():

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
module CrystalCoin
  module Consensus
    ...
    
    private def parse_chain(node : String)
      node_url = URI.parse("#{node}/chain")
      node_chain = HTTP::Client.get(node_url)
      node_chain = JSON.parse(node_chain.body)["chain"].to_json

      Array(CrystalCoin::Block).from_json(node_chain)
    end

    private def valid_chain?(node_chain)
      previous_hash = "0"

      node_chain.each do |block|
        current_block_hash = block.current_hash
        block.recalculate_hash

        return false if current_block_hash != block.current_hash
        return false if previous_hash != block.previous_hash
        return false if current_block_hash[0..1] != "00"
        previous_hash = block.current_hash
      end

      return true
    end
  end
end

In parse_chain():

We send a GET request to the /chain endpoint of the target node using HTTP::Client.get.
The /chain JSON response is parsed using JSON.parse.
We extract an array of CrystalCoin::Block objects from the JSON data using Array(CrystalCoin::Block).from_json(node_chain).

Crystal offers various JSON parsing methods. We’ll leverage the convenient JSON.mapping(key_name: Type) functionality, which provides:

Class.from_json: Creates an instance of the class from a JSON string.
instance.to_json: Serializes an instance of the class into a JSON string.
Getters and setters for the defined keys.

We need to define JSON.mapping within the CrystalCoin::Block object and remove the property usage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
module CrystalCoin
  class Block
   
    JSON.mapping(
      index: Int32,
      current_hash: String,
      nonce: Int32,
      previous_hash: String,
      transactions: Array(Transaction),
      timestamp: Time
    )
    
    ...
  end
end

In Blockchain#valid_chain?, we iterate through each block and:

Recalculate the block’s hash using Block#recalculate_hash and verify its correctness:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
module CrystalCoin
  class Block
    ...
    
    def recalculate_hash
      @nonce = proof_of_work
      @current_hash = calc_hash_with_nonce(@nonce)
    end
  end
end
  

Ensure each block is correctly linked to its predecessor via the previous_hash.
Check if the block’s hash meets the difficulty criteria (two leading zeros in our case).

Finally, we implement the /nodes/resolve endpoint:

1
2
3
4
5
6
7
get "/nodes/resolve" do
  if blockchain.resolve
    "Successfully updated the chain"
  else
    "Current chain is up-to-date"
  end
end

Congratulations! We’ve built a basic decentralized blockchain. The complete code is available on GitHub: final code.

The project structure should now look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
crystal_coin [master●] % tree src/
src/
├── crystal_coin
│   ├── block.cr
│   ├── blockchain.cr
│   ├── consensus.cr
│   ├── proof_of_work.cr
│   ├── transaction.cr
│   └── version.cr
├── crystal_coin.cr
└── server.cr

Testing Our Decentralized Blockchain

Set up two nodes on your network, either on different machines or using different ports on the same machine. I’ll use http://localhost:3000 and http://localhost:3001 as my two nodes.
Register the second node’s address with the first node:

1
2
crystal_coin [master●●] % curl -X POST http://0.0.0.0:3000/nodes/register -H "Content-Type: application/json" -d '{"nodes": ["http://0.0.0.0:3001"]}'
New nodes have been added: Set{"http://0.0.0.0:3001"}%

Add a transaction to the second node:

1
2
crystal_coin [master●●] % curl -X POST http://0.0.0.0:3001/transactions/new -H "Content-Type: application/json" -d '{"from": "eqbal", "to":"spiderman", "amount": 100}'
Transaction #<CrystalCoin::Block::Transaction:0x1039c29c0> has been added to the node%

Mine this transaction into a block on the second node:

1
2
crystal_coin [master●●] % curl http://0.0.0.0:3001/mine
Block with index=1 is mined.%

At this point, the first node has only the genesis block, while the second node has the genesis block and the newly mined block:

1
2
crystal_coin [master●●] % curl http://0.0.0.0:3000/chain
{"chain":[{"index":0,"current_hash":"00fe9b1014901e3a00f6d8adc6e9d9c1df03344dda84adaeddc8a1c2287fb062","nonce":157,"previous_hash":"0","transactions":[],"timestamp":"2018-05-24T00:21:45+0300"}]}%

1
2
crystal_coin [master●●] % curl http://0.0.0.0:3001/chain
{"chain":[{"index":0,"current_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","nonce":147,"previous_hash":"0","transactions":[],"timestamp":"2018-05-24T00:21:38+0300"},{"index":1,"current_hash":"00441a4d9a4dfbab0b07acd4c7639e53686944953fa3a6c64d2333a008627f7d","nonce":92,"previous_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","transactions":[{"from":"eqbal","to":"spiderman","amount":100}],"timestamp":"2018-05-24T00:23:57+0300"}]}%

Let’s resolve the first node to update its chain:

1
2
crystal_coin [master●●] % curl http://0.0.0.0:3000/nodes/resolve
Successfully updated the chain%

Verifying the first node’s updated chain:

1
2
crystal_coin [master●●] % curl http://0.0.0.0:3000/chain
{"chain":[{"index":0,"current_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","nonce":147,"previous_hash":"0","transactions":[],"timestamp":"2018-05-24T00:21:38+0300"},{"index":1,"current_hash":"00441a4d9a4dfbab0b07acd4c7639e53686944953fa3a6c64d2333a008627f7d","nonce":92,"previous_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","transactions":[{"from":"eqbal","to":"spiderman","amount":100}],"timestamp":"2018-05-24T00:23:57+0300"}]}%

Our Crystal blockchain is working flawlessly!

Conclusion

This tutorial covered the fundamentals of building a public blockchain using Crystal. You’ve learned to implement a blockchain from scratch and create a simple application for sharing information on the network.

We’ve built a functional blockchain that can be deployed on multiple machines to form a decentralized network. While our CrystalCoin is a simplified example, it lays the groundwork for exploring more complex blockchain concepts.

Remember, the code provided here is a starting point and is not production-ready. Use it as a guide for further exploration and experimentation.