I will walk through the setup cloudflare to use as nameserver, DNS record management, and SSL/TLS setup. There is plenty more at cloudflare like rate limiting as well.

Let’s get this out of the way first Domain Name System (DNS) is a way to translate domain names like blog.king-11.dev to its IP Address.

host blog.king-11.dev
blog.king-11.dev has address 172.67.164.94
blog.king-11.dev has address 104.21.66.209
blog.king-11.dev has IPv6 address 2606:4700:3033::6815:42d1
blog.king-11.dev has IPv6 address 2606:4700:3032::ac43:a45e

The resources I moved from multiple different providers included:

• Custom Domain

• Static Github Pages Website

• Dynamic NextJS Website

• Home Server Network Tunnel

Let’s dive into how my late night network shenanigans went.

Custom Domain

I had a great time just using github.io, vercel.app and netlify.app. These service providers are amazing and offer you a lot for free. But there were primarily two reason for my switch:

• I wanted to setup a cloudflare tunnel for my jellyfin server at flat so that I can binge watch my series and movies from anywhere in the world

• Having everything under a single umbrella helps in developing the brand as people remember it easily so why not?

Purchasing Domain

So after asking people what top level domain (TLD) should I buy and thinking a little about getting free ones. I unloaded my pockets and got myself king-11.dev from Namecheap.

The only issue was they didn’t provide INR payment options just like cloudflare so I had to use my OneCard as that provides a good forex rate of 1% only.

I could have bought the domain from cloudflare too they are the most transparent in terms of domain registration as they only ask for the registration charges and nothing else. But their paypal integration wasn’t working for me and namecheap was cheaper so went with it.

Nameserver

Nameserver is the server which contain DNS records for your domain. Think of it like a phonebook which contains your addresses, there can be many phonebooks but only one some will contain your details.

graph LR
  B[Browser] -->|domain| N[Nameserver] --> |ip address| B

To manage my custom domain on cloudflare I had to update my nameservers on namecheap and point them to the ones that cloudflare provided me.

To verify that nameserver is set properly we can use dig command

dig king-11.dev NS +short
mark.ns.cloudflare.com. ollie.ns.cloudflare.com.

Now the domain is in hands of cloudflare it can now easily create new DNS records for my domain name.

Security Settings

We can do additional setup to ensure that the content is transferred securely between all involved entities. To do this will enable full SSL (Secure Socket Layer)/TLS (Transport Layer Security). Under websites -> king-11.dev -> SSL/TLS, we would set my encryption mode to Full.

graph LR
  B(Browser) -->|🔒| C(Cloudflare) -->|🔒| O(Origin Server)

Furthermore, I can upgrade all the HTTP connection to HTTPS which returns content to user over a secure transport channel.

1. Under the SSL/TLS section, there is option of Edge Certificates

2. Under Edge Certificates, we can enable “Always Use HTTPS”

graph TD
  B(Browser) -->|HTTP Connection| C(Cloudflare)
  C -->|307 Internal Redirect| B
  B -->|Connection Redirect| B
  B -->|HTTPS Connection| C

Github Pages

I have been using github pages to deploy my static sites which included my blog repository. It uses a github action to build and then publish my hugo based site onto github pages.

The change I needed to make here was

1. Add the custom domain in Settings -> Pages section.

2. Create DNS entry in cloudflare to point blog.king-11.dev towards a github server.

But I don’t know the server neither does cloudflare’s nameserver and given I am no more using king-11.github.io/blog that entry would just redirect to my custom domain. So who will find me the ip address for my site?

CNAME Entry

Given github is the only one aware about the IP Address of my server, cloudflare needs to add a mapping such that dns resolution for my subdomain is offloaded to github.

This is exactly what CNAME records provide a DNS level redirection, they map one domain to another domain where both can be handled by different nameservers.

graph LR
  B[(Browser)] -->|blog.king-11.dev| C[(Cloudflare)]
  C -->|blog.king-11.dev| G[(Github)]

So in the above case when users asks cloudflare for blog site it redirects user to github which given the custom domain configuration settings know which server this domain points to and can provide user the result i.e. an A Name Record (domain -> IP).

To see this in line action we can use the command

dig blog.king-11.dev +trace

Mixed Content Bug

After all this process it started to happen that the cover of my blogs wasn’t visible anymore. Checking the console of browser the issue was that

Mixed Content: The page at ‘https://blog.king-11.dev/posts/memory-management/' was loaded over HTTPS, but requested an insecure image ‘http://blog.king-11.dev/posts/memory-management/cover_hu3f9b19c18b96efe8ccfa749888a579de_415626_720x0_resize_q75_h2_box_2.webp'. This request has been blocked; the content must be served over HTTPS.

So browsers don’t allow accessing any HTTP (insecure) resource from an HTTPS (secure) page. But why was it using http?

I tried setting cloudflare to redirect http to https itself but browser was already blocking any http connection before even going out of the browser. I checked the generated html of my blog page to find the image component as

<img loading="lazy" srcset="http://blog.king-11.dev/posts/memory-management/cover_hu3f9b19c18b96efe8ccfa749888a579de_415626_360x0_resize_
q75_h2_box_2.webp 360w ,http://blog.king-11.dev/posts/memory-management/cover_hu3f9b19c18b96efe8ccfa749888a579de_415626_480x0_resize_q75_
h2_box_2.webp 480w ,http://blog.king-11.dev/posts/memory-management/cover_hu3f9b19c18b96efe8ccfa749888a579de_415626_720x0_resize_q75_h2_b
ox_2.webp 720w ,http://blog.king-11.dev/posts/memory-management/cover_hu3f9b19c18b96efe8ccfa749888a579de_415626_1080x0_resize_q75_h2_box_
2.webp 1080w ,http://blog.king-11.dev/posts/memory-management/cover.webp 1080w" sizes="(min-width: 768px) 720px, 100vw" src="https://blog
.king-11.dev/posts/memory-management/cover.webp" alt="" width="1080" height="1080">

So, the src tag is correct but image is loaded using srcset (due to precedence) whose urls are prefixed with http. Now I updated my baseURL in hugo config file but still nothing was happening generated html was still using http only.

So I checked further github without information that SSL/TLS is enabled on my system was passing http://blog.king-11.dev to my github action which wasn’t using baseURL set in config file. Finally, it made sense leading to a final commit with the following diff

     run: |
       hugo \
         --gc \
-        --minify \
-        --baseURL "${{ steps.pages.outputs.base_url }}/"
+        --minify
   - name: Upload artifact

Vercel Redirection

After going through github vercel setup was a breeze. Again given vercel deploys my site it has the A Name Record for it. So on cloudflare I just need to add a CNAME record pointing my root to vercel’s cname domain and rest vercel handles on its own.

Two key things useful for vercel is:

• Set SSL/TLS encryption to Full mode initially it is flexible, which can lead to ERR_TOO_MANY_REDIRECTS

• Enabling Always Use HTTPS allows ACME protocol used to validate the domain ownership to bypass restrictions and use http.

What came in handy here is vercel has a great documentation for bugs, troubleshooting and setup 🎉.

Cloudflare Tunnel

So this all started from cloudflare tunnels. I have added details in my post The Hacker Way of Living about how I use self hosting to keep a personal library of movies and series, downloaded using torrent and hosted on my raspberry pi 4 using jellyfin media server.

To access it umbrel provides tor based access which is very slow so I needed a faster solution. There was Tailscale VPN which wasn’t working so the other option was cloudflare tunnel.

graph LR
    B((Browser)) -->|jellyfin.king-11.dev| C[(Cloudflare)]
    O[(Origin Server)] --- D[cloudflared]
    D --> C
    M((👨‍💻)) --x D

So while there is a connection between cloudflare and cloudflared (d is for daemon process) on my raspberry pi, no other user can access it. I can then use cloudflare to serve it on a custom domain which I purchased.

We can go on our dashboard, Zero Trust -> Networks -> Tunnels to create a new tunnel. Now to expose it or even multiple other endpoints we need to add our public ip and what it internally maps to on the origin server.

Conclusion

So the idea was for me to easily access my jellyfin from anywhere in the world while my raspberry pi 4 easily sits at my flat. Sadly, my flatmate unplugged it and forgot to plug it back.

Screams!, first of all you never unplug a server without properly shutting it down it will lead to data corruption. And secondly the server always stays plugged in. I need to set some new flat rules and setup warnings signs. Alexa play “new rules” because I don’t have tickets to go and see dua lipa 😭.

That was all for this time, it was quite some learning about networks that my computer science course didn’t even care to touch. I hope when you try to setup your own custom domain or self host things this blog would come handy. Thanks for reading see you next time after my next trip :P.

Creating an empty vector is as simple as

let empty_vec: Vec<i32> = Vec::new();

But some questions remain unanswered if you haven’t tried manually doing memory operations like:

• how is a sized vector created using Vec::with_capacity?

• how is a vector resized on calling pop_back or push_back?

• and finally how is the memory freed when vector goes out of scope?

Let’s engage in old god C as its closest we get to actual system calls and try to figure out these question inside out.

Memory Allocation

To store a vector or object we need a backing address in the physical memory of our system. We use malloc to ask operating system for more memory which looks something like below.

int* numbers = (int *)malloc(8 * 42);

In above code we are trying to create an array of numbers or more pedantically a pointer to integer which is the starting point of our array.

The parameter to malloc is the number of bytes we want to allocate. Given on a 64 bit system an integer takes 8 bytes so we are allocating for 42 integers.

malloc returns a void * which is C’s way of saying it can be anything so we have to type cast it based on our requirement and understanding.

Dynamic Sizing

Generally we can’t rely on hard coding the size of objects integers might be simple but can still vary on a 32 bit machine. A more complex thing like a struct of multiple values will vary even more.

struct foo
{
  int element;
  struct foo *next;
};

So its better to use a utility function sizeof for calculating size of our type.

int* numbers = (int *)malloc(sizeof(int) * 42);

In case you are running on 512 MB system and system runs out of memory malloc will return NULL value.

It’s not the null you see in Java and other garbage collected languages, NULL means a zero value something like (void *)0 whereas the other keyword is used to identify a variable as not having any reference.

Pointer inside Pointer

How about we handle an even more complex scenario

struct wallet {
    int version;
    int *notes;
};

Now we have a dynamic array inside of a dynamic struct which we have to allocate memory for, using just sizeof will give an empty array to us and it will segfault if you try to access array’s memory.

struct wallet* w = (struct wallet *)malloc(sizeof(struct wallet));
// this will segfault
// w->notes[0] = 42;

So we have to allocate memory of the array of integers inside of our wallet.

wallet->notes = (int *)malloc(sizeof(int) * 42);

Zeroing Memory

If you use malloc the memory is allocated to us but its never zeroed out before we get it. It can have garbage values which if not cleaned can cause your program to behave in unexpected ways.

The solution for this is to use one shot allocation and memory zeroing using calloc.

w->notes = (int *)calloc(4, sizeof(int));

calloc allocates an array of size 4 where size of each element is sizeof(int). The returned memory is set to zero.

Now the question arises why not always use calloc:

• First it only works for array allocation so you can’t allocate a struct using it.

• It is slow, in case you don’t need to zero memory out and pass the buffer to some other function which maybe reads the file or does some other io just use the big brother malloc.

Resizing

To answer the next question that we had in regard of a vector is basically related to allocating more memory and de-allocating extra memory. We do that using realloc function.

We will declare our vector struct as below it will track backing store, current position and total size of store.

struct vector {
  int *backing_store;
  int current_position;
  int backing_store_size;
};

When we call push_back the vector checks whether the size of its backing store is sufficient or not. If not, it allocates a backing store of double size using realloc and the places the element there. The newly allocated memory will be uninitialised.

void push_back(struct vector *v, int element) {
  if (v->backing_store_size <= v->current_position) {
    v->backing_store_size = v->backing_store_size * 2;
    v->backing_store = (int *)realloc(v->backing_store, sizeof(int) * v->backing_store_size);
  }
  v->backing_store[v->current_position] = element;
  v->current_position = v->current_position + 1;
}

Similarly, we can handle pop_back, just one crucial optimisation we won’t reduce to half but instead if current position is less that 1/4th of backing store size then we reduce the backing store to quarter of its previous size.

All the functions above return a NULL value in case of error and set errno which is Cs way of communicating error kind.

Freeing

It’s also important that we keep our memory usage in check otherwise it can exceed the physical memory capacity really fast. We have to free unused memory and return it back to the operating system.

It’s done using free function call, you pass in the pointer where memory is allocated and it will return it back to the system.

int* numbers = (int *)malloc(sizeof(int) * 42);
free(numbers);

Common Errors

The rise of RUST memory safe languages, paradigm like smart pointers in C++ and system like RAII is because of the reason that there are so many memory issues that keep happening in systems.

The recent Crowd Strike one was also a memory issue where invalid memory buffer was being accessed.

Let’s go over some common gotchas that you have to keep in mind while managing memory.

Invalid Free

Accessing a memory which was already freed will cause SEGFAULT and trying to free (double free) it again is an undefined behaviour you might not wanna encounter.

Free expects a pointer from the result of call to malloc passing anything else will also result in erroneous program execution.

Less Allocation

It’s possible you might have allocated less memory than required or no memory at all for an operation in which case buffer overflow will happen or SEGFAULT.

It is one of the binary exploitation ways hackers use and will probably cost your service a lot of secret credentials.

char *src = "hello world";
char *dst = (char *)malloc(sizeof(char) * 4);
// dst has size 4 while required is strlen(src) + 1
strcpy(dst, src);

In above example we could have probably used strdup.

Garbage Values

Accessing garbage or uninitialised value without clearing your memory will result in undefined behaviour of your program. It’s sometimes good to use calloc or clear memory on your own.

Memory Leak

Allocated memory is returned back to operating system when program dies even if you don’t call free on your own.

But that’s only in case of short running processes, in case of long ones like databases and servers if you don’t free unused memory, the memory allocated to program will keep increasing and finally your system will run out of available memory leading to crash.

Finding memory issues

valgrind is one of the tools we can use to identify memory related issues. It can tell us about:

• uninitialised reads

• invalid memory read/write

• memory leak

• invalid free, and a lot more

For a buggy program like this

int *data = (int *)malloc(sizeof(int) * 100);
data[100] = 42;

It will tell us about the invalid write if you run it through and also about the lack of a free call.

gcc -g test.c -o test
valgrind --leak-check=yes ./test
# Invalid write of size 4
# 400 bytes in 1 blocks are definitely lost in loss record 1 of 1

In case we try to access uninitialised memory and run it through valgrind it will let us know

printf("value at index 11", data[11]);
# Use of uninitialised value of size 8

Conclusion

Knowing about memory management is very crucial to build safe and performance critical software which basically is every software.

Offloading tasks to garbage collectors in programming languages with automatic memory management does make it easy but things like memory leak can still happen there so you should know what’s happening in your program otherwise AI replacement is coming for you :P.

That was all for this week, I wrote this piece after reading through Operating System: Three Easy Pieces its a very nice book with exercises and references to research papers I am enjoying reading it will keep sharing my learning from it as I try to complete it fingers crossed.

You can read more of my articles on my blog.

Rate Limiting is a crucial part of designing any API it maybe private or public. A rate limiter caps how many requests a sender can issue in a specific window of time. A rate limiter provides following features:

• Defense against a Denial of Service from malicious actors trying to pull down the service.

• Ensures the load on servers is always under maintainable state by discarding too many requests.

• Prevent broke developers from receiving an unexpectedly large bill in today’s auto scalable cloud service offerings.

I can relate with you Max :’(

Algorithms

Rate Limiting implementation can use different algorithms based on the requirement of our applications. Two key parameters are how we define our user (ip address, user id, device, etc.) and the second being our time window (1 minute, 1 hour, etc.)

Token Bucket

In token bucket algorithm we have a fixed size bucket which keeps getting filled by tokens at a uniform rate. When a new request arrives it is only processed if we have at-least one token in the bucket, otherwise the request is dropped.

flowchart LR
    r(Request)
    t[[tap]]
    b[(Bucket)]
    c[/count > 0/]
    p((Process))
    d((Drop))

    r --> c
    t --> b
    b <--> c
    c -->|true| p
    c -->|false| d

Leaky Bucket

In leaky bucket algorithm there is a fixed size queue where new requests are queued. These requests are then consumed at a uniform rate. A new request is dropped if bucket is a it’s capacity.

flowchart LR
    r(Request)
    b[(Queue)]
    c[/!capacityFull/]
    p((process))
    d((drop))

    r --> c
    c -->|true| b
    c -->|false| d
    b --> p

Leaky and Token bucket algorithm are relatively simple to implement but on the downside it’s difficult to find the right value for bucket size and token generation/request consumption rate.

Fixed Window Counter

In fixed window counter we keep track of number of requests within a fixed time range e.g. a minute or 30 seconds etc. We only process a limited number of requests within the time window beyond which the requests are dropped (red ones).

gantt
    title Requests v/s Time
    dateFormat m:ss
    axisFormat %M:%S
    tickInterval 10second
    R1 :0:00,10s
    R2 :10s
    R3 :0:25,10s
    R4 :crit,0:40,10s
    R5 :crit,0:45,10s

The issue with fixed window counter comes around when there are requests coming near the boundary of two windows. In such a scenario we actually end up processing more request within time window than set parameter.

gantt
    title Requests v/s Time
    dateFormat m:ss
    axisFormat %M:%S
    tickInterval 10second

    R3 :0:30,10s
    R4 :0:40,10s
    R5 :0:45,10s
    R6 :1:00,10s
    R7 :1:00,10s
    R8 :1:20,10s

In the above example within time range of 00:30 - 01:30 instead of 3 the rate limiter allowed 6 requests which is double of what was expected of it to allow in a minute. O(2*n)

Sliding Window Log

Sliding Window log is a dynamic one which handles the previous faults in fixed window counter by keeping track of requests in the current time window which is updated with every new request. It also uses a log to keep track of current requests.

Whenever a new request arrives

• We delete all the entries older than current time window.

• Now request’s timestamp is added to our log which now represents all the requests made in current time window.

• If the len(log) <= rate then we process the request else its dropped

Generally redis sorted sets are the used to keep track of timestamps in an efficient data structure which allows easy querying and updates.

Even with all its goodness its a little flawed due to the high memory usage. In case of a DoS attack we can have thousands of these log entries leading to huge memory usage and no valid requests will be processed in this duration these drawbacks lead us to the last algorithm.

Sliding Window Counter

It is a hybrid approach inspired from fixed window counter and sliding window log. It takes the best part of both and combines them sliding window log + fixed window counter.

To fix the flaws of fixed window counter it uses a weighted counter to smooth out burst around window boundaries which takes previous and present window requests into account.

percent = (cur_time - start_time_cur_window) / window_size

weighted_counter = requests_prev_window * (1.0 - percent) + requests_cur_window * percent

Liked sliding window log we need to use memory for keeping track of history. To count requests from each sender we will use multiple fixed time windows 1/60th the size of our rate limit’s time window like figma does. This remedies the memory footprint issue limiting it to just 60 values for every user.

4 bytes per counter * 60 counters * 100,000 users = 24,000,000 bytes OR 24MB

To reduce our memory footprint, we can store counters in a Redis hash which are highly efficient in their storage usage. Each new request which increments the counter it can set the expiry for it based on our time window but still there is a possibility of DoS attack to remedy this we should clean up hashes at regular intervals if they are rapidly increasing.

flowchart LR
    r[request]
    i(increment counter)
    c[/weighted counter < limit/]
    re[(Redis)]
    p((process))
    d(drop)

    r --> i
    i --> c
    c <--> re
    c --> p
    c --> d

HTTP Response

Now that we have rate limited user its also important to convey it back to the end user. HTTP 429 - Too Many Requests is the status code which is created for this purpose. We can add in more details in HTTP Headers as well

• X-Ratelimit-Remaining

• X-Ratelimit-Limit

• X-Ratelimit-Retry-After

I wrote this blog to better my understanding as am currently reading the holy grail of system design: The System Design Interview Book. I was never rate limited before but recently when I unfollowed over 1k people on Instagram it rate limited me.

I made use of mermaid diagrams for the first time in my blog compared to using excalidraw. Mermaid is pretty cool, easy to use, keeps me inside the markdown and its just text so very easily transferable.

I hope I was able to share my understanding with you, thanks for your support till here. If you’ve enjoyed this read, please share it, to help others find it as well. Also, feel free ping me up for any comment on @1108King😁

There was one space which was left for me to explore with open source, Bitcoin. Summer of Bitcoin started when I did my GSoC and the next summer (2022) again I didn’t apply because I was focused on my internship at Microsoft to grab that P.P.O. and chill for final year.

I was by no chance a bitcoin supporter I had heard news about bubble crash, people losing so much money including my own maternal uncle, the energy consumption for mining activities and then the associated latency of transactions. The idea seemed innovative but not at all practical to me.

I even jotted my thoughts and posted on whatsapp status. Two people (Nishant Kumar and Aditya Sharma) I vividly remember came to help me understand Bitcoin better and work towards getting a better understanding of its internals before making random claims. So that’s what I aimed to do with summer of bitcoin grab understanding of bitcoin better and completing the holy grail of open source programs before ending my college journey.

Application Process

I went ahead and applied on website for summer 2023 very late as I was engaged with Inter IIT Tech 11.0. Before getting entry on the discord server for proposal round we had to complete two bitcoin courses by saylor this included going through all the content and then giving a final exam. With whatever minimal understanding I had and internet resources, I gave the test after it was done I was on the SOB discord server.

Project Selection

With my new found love for rust I wanted to apply into a project which was written in rust. To my surprise rust was amongst the predominant technology being used in bitcoin which speaks a lot about the intent to make sure bitcoin and its associated projects are secure software to use.

While most projects were already flooded with applicants VLS caught my eye as there weren’t many people engaged so I went with it. I was welcomed by Ken who is such a humble individual with so much knowledge and experience.

Competency Test

Every project in the organisation had an associated competency test. We had to build the software and like every new software I couldn’t at first. It’s generally because I use bleeding edge linux like fedora and manjaro (in past) while it takes sometime for packages to catch upto them.

make build-experimental
# error: not a problem with poetry but with grpcio (1.47.0)
# latest gcc wasn't yet ready to work with grpcio

I didn’t want to setup a virtual machine as they are very slow. Also I didn’t want to shift to ubuntu whose operations were working fine. So I created a ubuntu container and tried running vls tests inside it. I encountered some errors related to lowdown which ken helped me resolve.

Again I couldn’t do this because containers don’t support systemd.

flowchart LR
    s[systemd]
    sn[snapd]
    btc[bitcoind]
    s --> sn
    sn --> btc

I think I tried WSL but systemd isn’t directly supported there as well. I was only able to make VLS work on WSL later around August and documented my journey with WSL in a github issue

Finally I made some space and triple booted my beast of a laptop. So I had Ubuntu 22, Fedora 35 and Windows 10 on the same device now. Finally I completed competency test which was to find the time taken by some of the operations by wrapping function calls in Instant::now() and elapsed() on certain tests.

Time elapsed in sign_counterparty_commitment_tx_phase2() is: 1.430216ms
Time elapsed in validate_commitment_tx() is: 30.734µs

There was only one project for VLS. The bench-marking one:

The project aims to benchmark and improve the performance of the Validating Lightning Signer.

I clarified on the expectations of project from ken and devrandom:

1. Identifying main operations for benchmarking

2. Benchmarking the main operations of a node running against the VLS

3. Identifying bottlenecks causing latency for different configurations

4. Updating the code base to handle those bottlenecks

I covered details about two options for creating bench-marking tests criterion and test. Listed down ways we can do optimisation algorithmic, config, inlining functions, etc. During phase of proposal submission I started helping other beginner contributors as much as I can while trying to understand the project.

Last Minute Submission

I delayed the whole writing of proposal till very late. I don’t remember why I did it but it was probably because it was my final semester in college and I wanted to leave no stone un-turned explore everything, hangout with friends, work for council and clubs, etc.

The deadline for submission was 16th April, 7:30AM and I began writing my proposal after coming from Shubhanshu and Yamini’s birthday party on 15th April at 11:50PM. I somehow kept myself awake through the process of writing proposal and submitted the proposal. With no expectation of getting selected or not I was at peace.

Selection

Jack contacted me while they were checking my application to know about when do I start with Microsoft. I misread and talked about my internship experience instead, he told me they had submitted results and Adi will reach out to us.

Summer of Bitcoin

I had the onboarding call with ken, devrandom and jack. There was one more fellow intern with me dhruv. The team liked both of our applications and hence decided to increase the slots and create one more project for memory usage bench-marking as well. I took the time latency and dhruv memory.

The team got us to buy STM32F413H devices which would come in handy for our projects.

Merge Requets

“Talk is cheap. Show me the code.” - Linus Torvalds

Documentation

Documentation: Apart from this I worked towards fixing documentation which is the first part any developer encounters making it easy so that others don’t face the issues I did. There were changes in packages, nightly version to use for STM and minor things.

Fedora Setup: The first thing I did as part of project was to make life easier by working out details on how to make vls work again on fedora linux. I added the details for the same into our updated documentation.

Operation Latency

We started with our decided plans of first doing the operations (keysend, pay, etc.) and checking their latency with multiple configurations of device (socket server/stm32) and logging modes (debug/info).

Below is the flow when any operations happen on a lightning node with VLS.

sequenceDiagram
    participant SignerA
    participant NodeA
    participant NodeB
    participant SignerB

    activate NodeA
    NodeA->>+SignerA: SignCounterpartyCommitment(m+1)
    SignerA-->>-NodeA: ack

    NodeA->>+NodeB: commitment_signed(m+1)

    NodeB->>+SignerB: ValidateHolderCommitment(m+1)
    SignerB-->>-NodeB: ack

    NodeB->>+SignerB: RevokeHolderCommitment(m)
    SignerB-->>-NodeB: ack

    NodeB->>-NodeA: revoke_and_ack(m)

    NodeA->>+SignerA: ValidateCounterpartyRevocation(m)
    SignerA-->>-NodeA: ack

    deactivate NodeA

For running the operations using socket server I had setup a testnet node on my laptop which was connected to ken’s node. I had to manually go through the logs and ask ken to provide me the same using which I created diagrams with time for each operation.

Like a regular programmer who denies any form of manual labour I tried automating the task and making my life a little easier using scripts which used to:

• copy the vls log file and then truncate it to 0

• do the operation of keysend and pay

• wait for sometime after the operation is done

• copy the log file again into current directory.

This helped reduce the manual labour somewhat by getting logs for precisely the time of operation.

I also did the operations for STM but instead of doing them with an external node I utilised the system tests which create temporary nodes on the same system.

Benchmarking Tests

After we had done these operations for sufficient time and were happy with the results. I started working on writing benchmark our three major operations.

• Sign Remote Commitment

• Validate Holder Commitment

• Validate Revocation

I also added in benchmarks for our secp256k1 since we have a lot of activities using its functions. Finally we had a good overview on our operation latency.

7609 bench INFO - Bench: Sign ECDSA, 200 iterations in 3360ms, each 16.8ms
12941 bench INFO - Bench: Verify ECDSA, 200 iterations in 3521ms, each 17.605ms
12966 bench INFO - Bench: SHA256, 200 iterations in 16ms, each 0.08ms
17941 bench INFO - Bench: Secp256k1 Create, 200 iterations in 3316ms, each 16.58ms

Sled Issue

During the testing devrandom found it odd that even on a rotating disk the performance tests for persister were running really fast. Going further it was realised that it was issue of our key-value store sled. It was not have been syncing to disk correctly.

Sled uses sync_file_range for flushes, which does not guarantee durable writes, except on specialised hardware (e.g. battery backed-up HD cache). Hence, we decided to move away from sled to redb.

Takeaways

The biggest key thing for me was I understood bitcoin why it was created, how will it go further and what will take it go further. I poured all my understanding into this blog Bitcoin, the Financial System.

Along the way I got to interact with highly skilled and amazing mentors Ken and Devrandom. By reviewing my MRs and having discussion with me around them they helped me get a better perspective on software development, testing, security etc.

I have continued contributing to VLS even became one of the core contributors with ownership of the VLS Container project. I have been invited to bitcoin++ austin! as a speaker, I am finding it hard to get a US Visa but hopefully I might get a chance to visit there, if you know someone who can help please do reach out to me.

Thanks for reading like always I am starting to get back in the routine of writing one blog per week hoping to write more and learn more.

Struct

Our bare minimum struct to represent an operating system contains a few common fields that would exist in each of the operating system.

typedef struct {
  char organization[];
  int releaseYear;
  float version;
  int is_opensource;
} os;

It’s missing one thing though the type of operating system like macOS, linux, windows, etc.

Enum

We will add in a new field into our structure to represent the type of operating system by using an enum.

typedef enum
{
  Linux,
  Windows,
  MacOS
} os_type_enum;

typedef struct {
  ...
  os_type_enum os_type;
} os;

More Data Fields

Consider the case of Linux, there are many distributions like Manjaro, Fedora, Ubuntu, Zorin, etc. we want to store this data as well.

So what can we do now with our struct and enum combination is we can create a new field which will be nullptr in case of MacOS and Windows and in case of Linux it will hold data.

typedef struct {
  ...
  char distribution[];
} os;

Now it comes down to user to manage this logic properly the language won’t provide support for anything. But there is a better way to handle such things we can use union.

Union

C language, yes the elder one has features for us to easily create sum type by using union keyword. And you thought your new and shiny language has an advantage over it “much to learn you still have”

We will create three new structs each representing meta details about each operating system. In case of macOS and Windows these would be empty structs but in case of linux will have a data field.

typedef struct {
  char distribution[];
} linux_details;

typedef struct {} mac_os_details;
typedef struct {} windows_details;

union os_details_union {
  linux_details linux;
  mac_os_details mac_os;
  windows_details windows;
};

typedef struct {
  ...
  os_details_union os_details;
} os;

Union forms the basis for sum types which is actually a mathematical concept represented by A U B where A and B are individual types.

The opposite of sum type is product type, which is represented in programming languages by a normal class or struct. An interface represents an intersection in mathematical terms.

Memory Requirements

A union variable will always be the size of its largest element. So even if I have some fields in the other two structures as well it still won’t be an addition of individual struct size but instead a max.

int struct_memory = sizeof(linux_details) + sizeof(mac_os_details) + sizeof(windows_details)
int union_memory = max(sizeof(linux_details), max(sizeof(mac_os_details), sizeof(windows_details)))

While writing this blog I came to know about this benefit which is utilized in low resource devices.

Pros and Cons

Now the union logic isn’t on user but instead on the language runtime itself to manage. It brings in safety and clarity of usage to the end user of language i.e. the programmer but it also adds in un-safety around same memory utilization for all the fields.

What do I mean by memory un-safety? let’s consider below example of memory overwrite.

union random_data {
  int happiness_index;
  int sadness_index;
} data_holder;

data_holder.happiness_index = 5;
// below assignment will override happiness_index
data_holder.sadness_index = 2;
printf("happiness_index is %d and sadness_index is %d", data_holder.happiness_index, data_holder.sadness_index)
// happiness_index is 2 and sadness_index is 2

As we can see a wrong usage of union can cause loss of data as only one field can exist in the union because they share same memory space. This is a simple example of memory overwrite, in a mix of structs and primitive data types we face a even bigger threat.

Safe Sum

There are other languages as well which provide union/sum types with memory safety like Haskell, OCaml, etc. but I would like to go with Rust here as am familiar with it.

In rust there is no special union type instead enum values can hold the data we need and it make our lives easy.

enum OsTypeEnum {
  Windows,
  Linux(String),
  MacOS,
}

struct Os {
  os_type: OsTypeEnum,
  organization: String,
  release_year: i32,
  version: f32,
  opensource: bool,
}

That’s it so simple so elegant just looking like a wow.

Pattern Matching

While in C even with union type we still had to rely on the os_type_enum to check what operating is the os struct representing and a user error there would result in memory issues.

Modern languages have pattern matching which allows them to match against the structure of types, both complex and simple in a very clean manner.

match os_type {
  OsTypeEnum::Windows => print!("Hail Microsoft!"),
  OsTypeEnum::MacOS => print!("richie rich :P"),
  OsTypeEnum::Linux(description) => print!("never ending suffering {}", description)
}

In rust we can pattern match against things like:

• Literals

• Destructured arrays, enums, structs, or tuples

• Variables

• Wildcards

• Placeholders

You can read more about the capabilities of rust specific pattern matching in the rust book.

Other Languages

Not every language has sum type or pattern matching but most of the languages are moving towards enabling it, not in a true manner but something is better than nothing coughs in golang.

C Sharp

It has become a language that I have started liking a lot. C# has no true equivalent to union type like C or Rust but we can provide similar feature using inheritance and pattern matching.

I took reference from a blog which goes in detail about improvements made till C#9 to allow below example with a very clean syntax.

record OsType {
  public record Windows : OsType;
  public record Linux(string distro) : OsType;
  public record MacOS : OsType;
  private OsType() { } // private constructor can prevent derived cases from being defined elsewhere
}

C Hash Sharp also provides pattern matching abilities using the switch statement but it won’t be able to do an exhaustive match.

osType switch {
  OsType.Windows windows => "its updating",
  OsType.Linux linux => $"where are the display drivers: {linux.distro}",
  OsType.MacOS macOS => "flexing that half eaten apple",
  _ => throw new ArgumentException(message: "invalid OsType value", paramName: nameof(osType)),
};

Java

In Java we have records, pattern matching and sealed interfaces as part of Java17 which can allow us to implement union types.

public sealed interface OsType {}
public record Linux(String Distribution) implements OsType {}
public record MacOS() implements OsType {}
public record Windows() implements OsType {}

For pattern matching we can use the switch and isInstanceOf to do exhaustive matching with a sealed interface.

switch (osType) {
  case LinuxInformation linux -> "I use arch btw";
  case MacOS macOS -> "loan installment pending";
  case Windows windows -> "even dad can use it";
}

Kotlin

My new found love does union types mostly like java but instead of interface we use a sealed class, data objects and data classes. Data Class seems superior to Java records as they can extend other classes whereas a record can’t.

sealed class OsType private constructor() {
  data class Linux(val distribution: String) : OsType()
  data object MacOS : OsType()
  data object Windows : OsType()
}

The pattern matching happens using when

when (os) {
  // smart casting happening here
  is Os.Linux -> "mr. robot uses $(os.distribution)"
  is Os.MacOS -> "no more space left"
  is Os.Windows -> "its AI everywhere"
}

Conclusion

Union/Sum Types are concrete types even in the face of diverse options. C introduced it to us long back when it was a crude mathematical model and it has been improved thoroughly now with time and is visible in modern languages like Rust, OCaml, Scala, etc.

The old gods like C#, Java, etc. are trying to keep up with trendy types by adding more new feature but it just doesn’t feel as good. I was even planning to show example for doing it with Go but it just can’t do that in a humanely readable way.

That was all for this week, thanks for reading, stay tuned on the Backpacking Dream and Jai Shree Ram!!.

A coroutine is an instance of a suspend-able computation. It is conceptually similar to a thread, in the sense that it takes a block of code to run that works concurrently with the rest of the code.

However, a coroutine is not bound to any particular user level thread. It may suspend its execution in one thread and resume in another one. This scheduling of coroutines is handled by the Kotlin dispatchers similar to go runtime

Coroutines are less resource-intensive than JVM threads. Code that exhausts the JVM’s available memory when using threads can be expressed using coroutines without hitting resource limits.

Suspending Function

Suspending functions are at the heart of coroutines. They can use other suspending functions (like delay in this example) to suspend execution of a coroutine and free the thread for useful computation by other coroutines.

suspend fun doWorld() {
    delay(1000L)
    println("World!")
}

Coroutine Builders

To actually build a coroutine we need to use coroutine builders which cater to multiple different scenarios where coroutines can be used.

Launch

A launch coroutine builder returns a Job object that is a handle to the launched coroutine and can be used to explicitly wait for its completion.

launch { // launch a new coroutine
    delay(1000L) // non-blocking delay for 1 second (default time unit is ms)
    println("World!") // print after delay
}

CoroutineScope

A coroutineScope builder can be used inside any suspending function to perform multiple concurrent operations

// Concurrently executes both sections
suspend fun doWorld() = coroutineScope { // this: CoroutineScope
    launch {
        delay(2000L)
        println("World 2")
    }
    launch {
        delay(1000L)
        println("World 1")
    }
    println("Hello")
}

The main difference from runBlocking method is that it blocks the current thread for waiting, while coroutineScope just suspends, releasing the underlying thread for other usages. Because of that difference, runBlocking is a regular function and coroutineScope is a suspending function.

Produce

We can also use produce builder where a coroutine is producing a sequence/stream of elements (very common in producer consumer patterns)

// infinite stream of integers starting from 1
fun CoroutineScope.produceNumbers() = produce<Int> {
    var x = 1
    while (true) send(x++)
}

Note: All functions that create coroutines are defined as extensions on CoroutineScope, so that we can rely on structured concurrency to make sure that we don’t have lingering global coroutines in our application.

Async

There is another builder, async it is quite similar to launch. It creates a single coroutine but instead of forgetting about return value like the case with launch returned Job, async returns a Deferred (non-blocking future) that represents a promise to provide result using .await().

val one = async { doSomethingUsefulOne() }
println("The answer is ${one.await()}")

Optionally, we can make async lazy as follows:

val one = async(start = CoroutineStart.LAZY) { doSomethingUsefulOne() }
one.start() // start lazily

Now that we have a concurrent environment where multiple coroutines are working simultaneously, we need mechanisms to synchronize data access/communication.

Channels

Channels are communication primitives that allow data to be passed between coroutines. One coroutine can send some information to a channel, while another can receive that information from it in a thread safe manner.

Channel is represented by three different interfaces: SendChannel, ReceiveChannel, and Channel, with the latter extending the first two.

There are multiple kinds of channels:

• Unlimited Channel

• Buffered Channel

• Rendezvous Channel

• Conflated Channel

  val rendezvousChannel = Channel() val bufferedChannel = Channel(10) val conflatedChannel = Channel(CONFLATED) val unlimitedChannel = Channel(UNLIMITED)

Note: Both Channel factory function and produce builder take an optional capacity parameter to specify buffer size.

Sending and Receiving

A Channel is conceptually very similar to BlockingQueue, except that its methods aren’t blocking and instead of suspending nature.

val channel = Channel<Int>()
launch {
    // this might be heavy CPU-consuming computation or async logic, we'll just send five squares
    for (x in 1..5) channel.send(x * x)
}
// here we print five received integers:
repeat(5) { println(channel.receive()) }
println("Done!")

Channel send or receive functions can suspend when the channel is full or empty respectively. The channel can be full if the channel size has an upper bound.

Closing

The producer can close a channel to indicate that no more elements are coming. Conceptually, a close is like sending a special close token to the channel.

A receiver iterating on channel stops as soon as this close token is received, so there is a guarantee that all previously sent elements before the close are received.

launch {
    for (x in 1..5) channel.send(x * x)
    channel.close() // we're done sending
}
// here we print received values using `for` loop (until the channel is closed)
for (y in channel) println(y)

Patterns

We can use the channel data structure to implement multiple concurrency patterns frequently used in concurrent/parallel programming.

Producer-Consumer

Channels simplify implementation of producer consumer pattern in kotlin to a large extent with a large set of easy to use functions. We can use the produce builder on producer side and an extension function consumeEach, that replaces a for loop on the consumer side.

// producer builder channel
val squares = produceSquares()
// consumeEach utility
squares.consumeEach { println(it) }

Fan-out

Multiple coroutines may receive from the same channel, distributing work between themselves. Unlike consumeEach, for loop pattern is the safe option to use when multiple coroutines are receiving because a consumer written via consumeEach always consumes (cancels) the underlying channel on its normal or abnormal completion leaving other coroutines aborted as well.

Shared Data

Coroutines can be executed parallelly using a multi-threaded dispatcher like the Dispatchers.Default. Given the scenario of parallel data access we have to prevent unpredictable behaviour of data.

Volatile

There is a common misconception that making a variable volatile solves concurrency problem, because volatile variables just guarantee atomic reads and writes to the corresponding variable, but do not provide atomicity of a larger action (like addition, subtraction, etc.)

@Volatile // in Kotlin `volatile` is an annotation
var counter = 0

Thread Safe Data Structure

A thread-safe (aka synchronized, linearizable, or atomic) data structure provides all the necessary synchronization for the operations that needs to be performed on a shared state.

val counter = AtomicInteger()

This is the fastest and most versatile solution for this particular problem. It works for plain counters, collections, queues and other standard data structures and basic operations on them. However, it does not easily scale to complex state or to complex operations that do not have ready-to-use thread-safe implementations.

Thread Confinement

Thread confinement is an approach to the problem of shared mutable state where all access to the particular shared state is confined to a single thread.

// confine each increment to a single-threaded context
withContext(counterContext) {
    counter++
}

This code works very slowly, because it does fine-grained thread-confinement. In practice, thread confinement is performed in large chunks, e.g. big pieces of state-updating business logic are confined to the single thread.

Mutex

Coroutines have Mutex. It has lock and unlock functions to delimit a critical section. Mutex.lock() is a suspending function which does not block a thread. There is also withLock extension function that conveniently represents:

mutex.lock()
try { ... }
finally { mutex.unlock() }

// using withLock
withLock(mutex) {
    try { ... }
}

Cancellation

Just launching the coroutine isn’t sufficient, at times even the good humans of this world have to become a part of cancel culture set by Mr. Popatlal to cancel some long running background coroutine.

var job = launch {
    delay(500L) // i will sleep
}
delay(200L)
println("cancel culture ftw")
job.cancel() // ask it to cancel
job.join() // wait for it to complete

There is also a cancelAndJoin utility method.

Cooperative

Coroutine Cancellation is of cooperative nature, the cancelled coroutine has to agree with cancellation and stop the computation. If a coroutine isn’t actively checking for cancellation signal then it can’t be cancelled.

All the suspending functions in kotlinx.coroutines are cancellable, they actively check for cancellation signal and then throw a CancellationException when cancelled. So we shouldn’t catch CancellationException without intent of stopping the computation.

try {
    delay(500)
} catch (e: Exception) {
    // log the exception, might have been a `CancellationException`
    println(e)
}

Cancellable Computation

To ensure that the library or program that we write aligns with the cancellation requirements we can pursue below two options:

• Calling the yield suspending function which checks for cancellation at regular intervals inside the coroutine.

launch {
    while (true) {
        // large computation
        yield()
    }
}

• Checking the isActive extension property available inside the coroutine via CoroutineScope object

launch {
    while (isActive) {
        // large computation
    }
}

Conclusion

Kotlin is an amazing language I genuinely had fun writing code in it during 2023’s Advent of Code which I didn’t complete :’(. I have felt this feeling last with C# and Golang not rust. Let’s cover this thought about joy of programming in a well articulated blog later so stay tuned.

Thanks for reading, you guys keep me going I am planning to start publishing my blogs on Medium as well for better reach to increase my Luck Surface Area till then adios, Happy Makar Sankranti may the wind gods be with you :) and yeah you know where to find me @1108King

Bitcoin is a financial system created to serve a highly political intent, a free and uncensored network where all can participate with equal access. It is a mechanism of organizing people and computers to make one immutable, digital document.

Bitcoin Offerings

• Bitcoin doesn’t keep track of the identities of the people (or machines) that use it making it permissionless.

• It doesn’t increase the supply of bitcoins to meet demand making it deflationary and inviolate.

• It doesn’t force itself on anyone making it consensual.

• It doesn’t exclude anyone from using it making it inclusive.

It is well enough that people of the nation do not understand our banking and monetary system, for if they did, I believe there would be a revolution before tomorrow morning - Henry Ford

Alt Shit Coins

Bitcoin is the OG crypto it’s not like other shitcoins, that keep popping up everyday. Every f-king day these wannabe crypto enthusiasts create an alt coin (short for alternate coin) thinking it will boon someday…it won’t.

While blue-chip cryptocurrencies, such as Bitcoin and Ethereum, have robust infrastructure and fundamentals to back them up, shitcoins rely on short-lived hype and FOMO (Fear Of Missing Out) to build up value. In March 2023, Ethereum co-founder Vitalik Buterin sold $700,000 worth of shitcoins that were airdropped to him, crashing the shitcoins price down as they are mostly operating on large investor money.

Breaking Barriers

Bitcoin has no boundaries a person sitting in USA can easily and safely do transaction with person sitting in India without worrying about paying currency exchange fees, foreign remittance, etc and going through hassle of multiple financial channels. It is one true global currency that doesn’t divide people based on country but instead provides a simple, safe and secure mechanism to cater to all your transaction requirements.

Primitive Institutions

I was just starting to write this article and my father told me about the latest news where people defrauded banks in Jodhpur and double spent their actual funds due to a fault in their servers resulting in creation of money valuing around 850 crores out of thin air.

Double Spend aka Byzantine General’s problem was the first problem Bitcoin had to solve as a distributed finance system. In 2008, it was done by Satoshi Nakamoto using blockchain, still the supposedly safe and secure financial institution can’t protect our money and suffer from problem which was solved a decade ago.

Current financial institution are here to earn from you and not cater to your needs. They won’t stop badgering you until you lock in your money with them using multiple Ponzi schemes.

Whatever money you don’t hold isn’t your money. - My Dad

When the day comes they start crashing like in 2009, there is nothing that would be able to protect your money instead they take the route of locking your money from any kind of withdrawal to save their arses first. I highly recommend anyone to watch The Big Short to better understand how it all went down in 2009 due to fraudulent and delusional financial institution.

Blackrock named Jamie Dimon’s JPMorgan as their Authorized Participant for their potential Bitcoin ETF. Jamie Dimon LITERALLY said earlier this month that the “if I were the government I would BAN Bitcoin” because only criminals use it and JPMorgan says ETF optimism is overhyped. You see how two faced these bankers are?

Negative Inflation

A salaried individual in today’s world has to worry about thing like stocks, mutual funds, PPF, bonds, options, etc because we have to beat inflation and inflation is normal right your money loses power and value it holds over time, it’s meant to happen

But let me ask, why is it normal though why shouldn’t a 100 rupees note today hold the same value 10 years later?

Economists have identified several possible causes for inflation from rising wages to increased aggregate demand to an increase in the supply of money.

The fools minting out the cash don’t understand shit about the financial systems how would they most of them don’t posses any form of official college education/degree. They keep increasing the supply of money as and when they deem fit which results in overall paper bill losing its power.

Bitcoin saves its owners from all these worries, given the limited supply which keeps decreasing every 4 years to half (Bitcoin Halving) making it a scarce resource. It will keep deflating (currently at 164% per annum) at an exponential rate compared to any other asset like land, stocks, options, etc.

Government of Goons

Spend some time with Bitcoin. Learn it, challenge it, and use it. You can assume no government wants you adopting this system in any capacity, and for that reason alone it’s worth consideration by honest, moral, and industrious people - Erik Voorhees

Every form of government be it monarchical or democratic or communist want just one thing control because without control they aren’t anything. Bitcoin is beyond their control:

• They can’t control the transactions or even identify individuals involved in a transaction

• They can’t measure the capital value you are holding in your bitcoin wallets

• They can’t inflate bitcoin by minting more of it as they do with paper money

• They can’t impose state embargos on bitcoin moving across the countries

All the above cited reasons are what makes bitcoin an ideal currency for medium of exchange from a user’s perspective not governments.

I had almost finished writing my article when I learned that India’s Financial Intelligence Unit (FIU), a branch of the Finance Ministry, had issued a notice and blocked the domain of several bitcoin exchanges, making another attempt to undermine bitcoin’s supremacy.

Data Speaks

Let’s consider two measuring tools which give us idea of how poorly our governments manage country finances, after all financial education needs education and most of our illiterate parliamentarians don’t have any formal education.

Fiscal Deficit: A fiscal deficit is a shortfall in a government’s income compared with its spending. A government that has a fiscal deficit is spending beyond its means. The government has to borrow money to meet its expenses. The government’s debt increases over time as it continues to borrow.

Fiscal Debt: It is the total debt accumulated over years of deficit spending. It is the sum of all the fiscal deficits over the years. It is the total amount of money the government owes to its creditors.

Every Indian today is born with a debt of Rs. 1,57,594.93 (debt per capita) all thanks to the wonderful schemes put in place by our governments to provide freebies instead of generating valuable assets in terms of people and infrastructure.

Instant and Secure

I understand that you are not willing to wait a minimum of 10 minutes on chai point until your bitcoin transaction confirms. Neither would you be willing to send a large sum of money in some investment or purchase over the bitcoin network since it is public ledger visible to everyone.

Satoshi always knew this and in 2009 itself he proposed for creation of layer-2 networks over the main bitcoin network.

Lightning Network

If Bitcoin is to be used for everyday payments the transaction speed must increase and transaction costs must decrease drastically. The Lightning Network (also referred to as Lightning, or LN) is a scalability solution built on top of Bitcoin that allows users to quickly send and receive BTC with virtually no fees.

Lightning makes Bitcoin a viable option it can be thought of as Indian UPI system for Bitcoin just decentralized. The Lightning network uses smart contracts (HTLCs) and payment channels between two parties.

Note: Lightning network can have as many channels as it wants so TPS has no limit or we can say sky is the limit.

Liquid Network

The Liquid Network is a Bitcoin layer-2 solution enabling the fast, confidential settlement and issuance of digital assets, such as stablecoins, security tokens, and other financial instruments, on top of the Bitcoin timechain.

While it’s not fully permissionless like Bitcoin, Liquid is fault-tolerant, censorship-resistant, and has different trade-offs than other second layers like Lightning. Liquid’s high degree of privacy represents an important benefit for traders.

Conclusion

Further read to get a in depth understanding of bitcoin:

• The Bullish Case for Bitcoin | by Vijay Boyapati

• Why Bitcoin

• Bitcoin: A Peer-to-Peer Electronic Cash System

• Money Creation and Society

Give bitcoin a try, if you have any queries am more than happy to help you out with them at @1108King but like any other investment it’s a risk “should you choose to accept it your mission is to explore bitcoin”. Thanks for reading and have a great year ahead.

go f(x, y, z)

The evaluation of f, x, y, and z happens in the current goroutine and the execution of f happens in the new goroutine.

Goroutines run in the same address space, so access to shared memory must be synchronized. The sync package provides useful primitives, although you won’t need them much in Go as there are other primitives like channels.

When to use?

With goroutine, we can make asynchronous network IO and continue the program without the need to wait for the response to come back. Obviously, network IO is just one of the many examples. We can do parallel processing like a mini map reduce and fully utilize each CPU core we have on our modern days machine.

Architecture

goroutine is not a os/kernel thread. It is an abstraction of a thread, more commonly known as application thread. It is Go runtime’s job to manage how many OS threads to create and how goroutines are mapped to and executed on OS threads.

Go Runtime Components

There are three main components involved in the execution of a goroutine i.e. the goroutine, an abstract logical processor and the machine.

• Goroutine: Each goroutine is described by a G struct. The struct keeps track of various runtime information, like stack trace and status.

• Logical Processor: P stands for Logical Processor. It’s an abstract resource or context. We usually call P a context that is mapped to a M to run Gs.

• Machine: The struct M stands for machine, which maps to an OS thread which can execute a G or goroutine.

Entity Interaction

When a Go program starts, it is given a logical processor P for every virtual core. Every P is assigned an OS thread M. Every Go program is also given an initial G which is the path of execution for a Go program. OS threads are context-switched on and off a core, goroutines are context-switched on and off a M.

P is responsible for scheduling all runnable Gs on M. If and when an executing G makes a blocking call, P moves onto a different M while G blocks the current M.

Controlling Parallelism

We can set the maximum number of goroutines executing in parallel by limiting the number of P.

package main

import ( 
    "fmt" 
    "runtime" 
)

func main() { 
    fmt.Println(runtime.GOMAXPROCS(1)) 
}

This limits the maximum number of P to 1 but that won’t limit the program to not spawn more than 1 M (OS Threads) in case of blocking call as stated above.

Thread Synchronization

Don’t communicate by sharing memory; share memory by communicating

This is the moto which go lang has popularized, in which we don’t give threads access to shared memory but instead share messages/data by communicating with each other.

Channels

Channels are a typed thread safe conduit through which you can send and receive values with the channel operator, <-.

ch <- v // Send v to channel ch.
v := <-ch // Receive from ch, and
         // assign value to v.

(The data flows in the direction of the arrow.)

Like maps and slices, channels must be created before use:

ch := make(chan int)

By default, sends and receives block until the other side is ready. This allows goroutines to synchronize without explicit locks or condition variables.

We can also use buffered channels which won’t block on every operation but instead only on the absence of data.

Note: Channels aren’t like files; you don’t usually need to close them. Closing is only necessary when the receiver must be told there are no more values coming, such as to terminate a range loop.

Select Statement

The select statement lets a goroutine wait on multiple communication operations.

A select blocks until one of its cases can run, then it executes that case

select {
    case c <- x:
        x, y = y, x+y
    case <-quit:
        fmt.Println("quit")
        return
}

Shared Data

Go’s standard library provides mutual exclusion with sync.Mutex and its two methods:

• Lock

• Unlock

We can define a block of code to be executed in mutual exclusion by surrounding it with a call to Lock and Unlock.

c.mu.Lock()
// Lock so only one goroutine at a time can access the map c.v.
defer c.mu.Unlock()
return c.v[key]

Data Race

It’s important to use patterns like channels and mutex locks to prevent data races otherwise go has a data race detector which will panic at runtime.

When the race detector finds a data race in the program, it prints a report. The report contains stack traces for conflicting accesses, as well as stacks where the involved goroutines were created.

WARNING: DATA RACE
Read by goroutine 185:[[]]
  net.(*conn).Write()
      src/net/net.go:129 +0x101

Previous write by goroutine 184:
  net.setWriteDeadline()

Goroutine 185 (running) created at:
  net.func·061()
      src/net/timeout_test.go:609 +0x288

Goroutine 184 (running) created at:
  net.TestProlongTimeout()
      src/net/timeout_test.go:618 +0x298

Thanks for reading please share your reviews on twitter with me @1108King and if you found the article useful do share if for visibility.

Childhood Fun

Since my childhood I was taught by my cousin brother (Karan) on how to utilize the amazing peer to peer network named torrent we used to keep our systems running late at night to just download a video game, dark ages I know.

Back in time when I was worthy I even hard booted my first phone (YU Yureka 2014) with alternative android based operating system Cyanogen (renamed to LineageOS).

My father got me a second hand computer first to experiment on, I do remember installing my first Ubuntu/Linux Mint but didn’t quite understand it that time neither there were good online resources to help me so moved back to Windows. I used to break it down every other day and papa had to call Saini/Mahesh Uncle to fix and reinstall Windows.

Damn, those were good days (emotional moment), it was the time of flash sales we used to look up internet find scripts to run that would get us the product that we wanted quickly. Now after doing web development I can make those scripts myself I even help my father in auctions to get the contract in milli seconds using scripts.

Moving away from Subscription Hell

I will try to break down things and go service by on what are the possible alternatives. Possible being the keyword these aren’t the exact service of what the evil corporate gives you but it’s very much doable to live with the alternatives. Because of course nothing not everything comes for free.

YouTube Premium

Depending on platform that you are using Desktop or Android (yeah shu go away iPhone users) there are two solutions.

Brave Browser

If you want to listen to the beats while working on laptop like I usually do just start using Secure, Fast, & Private Web Browser with Adblocker | Brave.

Brave is the best browser probably if you are a privacy enthusiast and want to protect your data and habits aka yourself from the clutches of corporate data centres then use it.

• It is as fast as Chrome because under the hood they use Chromium only the engine that powers Chrome, Edge, etc.

• It blocks scripts, trackers, cookie banners, fingerprinting, third party ads, etc. and that’s why the sites don’t get stuck and frozen while browsing.

• It has its own privacy focused search engine with AI features as well. It isn’t as good as bing or google but well it boils down to making compromise because sometime you want privacy and sometimes you just want results.

• It enables https by default which keeps your data secure over the wire from other people who can listen to all the online chatter. So just install brave and head over to Youtube Music or Youtube to listen to your favourite songs without ad interruption

ReVanced

It’s ReVanced not Vanced before you start saying that Vanced was discontinued not because it was an ad blocker but it was due to logo infringement.

Here is the guide that I used to install revanced on my Android phone and it just works. If it doesn’t for you there are two options:

• Revanced is open source unlike Vanced so just go out there and contribute to the project by either providing a reproducible or if you can code just fix it.

• Pay up 👁️👁️

Spotify Premium

There are multiple modded versions of Spotify available which just work fine still even with all the restrictions. I got one from Ankit you can ask him to :P or just a quick search gives you a lot of modded Android version iPhone users shu shu definitely available.

Well there is definitely attached risk for installing mod maybe it is some harmful app instead but well isn’t that life :P. Everything is risk from stock markets to driving vehicle to travelling to Himachal but even the most chaotic of experiences won’t stop me at least from getting that adrenaline shot of investing in a new found fund, going at 170km/hr on Delhi - Mumbai Expressway, Returning to my beloved mountains on a trip to Nanital, etc.

Cloud Notes

I keep track of every single thought of mine blogs, shopping list, contacting people, good reads, healthy lifestyle, investments, etc literally everything. Partially I write down some of the urgent notes from meetings into diary as well.

I use Obsidian for writing down notes and thoughts it’s just amazing and you will only know it’s awesomeness after trying am even writing my blog in this. Now to sync it, I use Syncthing, it’s a cross platform app to keep files and folders between multiple devices in sync. I use it to keep my desktop and android obsidian in sync. Weekly I upload it to my self hosted Next Cloud so that it stays good.

Obsidian comes with Obsidian sync as well which is $10/month it’s costly so I definitely would not recommend it until you are not willing to make above efforts. You also don’t have to use nextcloud you can just use OneDrive or similar alternatives as well.

Self Hosting

Now this is the part am most excited about. I recently got a Raspberry Pi 4 and installed Umbrel OS, what umbrel offers is a plethora of self hosted alternatives to your regular cloud hosted things like which are just a single click away on their app store

I personally use all of them except PhotoPrism (Dad paid for Google One his mailing quota was going above 20GB limit) and LlamaGPT (I have access to Bing Chat Enterprise running on GPT 4 thanks to Microsoft 😼).

I can access my umbrel on my Local Area Network (LAN) which is provided by my Home WiFi Router through any web browser by going to https://umbrel.local . It’s possible to access it remotely using Tor Address but that is a little slow but comes useful in case I have to show document which isn’t available on my mobile.

Vendor Lock In

When you use cloud hosted services you become at most of the times a victim of vendor lock in. As I wrote in my previous blog about Loss of Customer Empathy | Backpacking Dream the corporate world is more focused on locking you into their product rather than allowing you to easily move away from their service/product.

I discussed it in Microsoft with one of our CVP PM about vendor lock, he also agreed to the fact that it is everywhere but there is additional value attached with it for users using it for long periods of time so sometimes it’s not bad. I concur, sometimes its good but it becomes bad when Google Photos won’t even provide you with a delete all button 😿

Self hosted alternatives generally use an open source protocol/standard which allows you to easily move away from it if you aren’t able to gather value out of it.

Can’t we just pay?

Let’s calculate the bare minimum cost of self hosting:

Obviously this is the basic I personally use HDD -> SSD (2 TB) and Basic Case -> Metallic Heat Sink Case but that is just me you don’t have to it varies on your use case. Now that is all until my hardware fails (generally SD Card) I have to get a new one but that would be all.

Let’s compare to available cloud solutions calculated at an yearly cost.

So a self hosted solution within two years will be more cost efficient than a cloud solution, with additional perk of no vendor lock in.

Azure Student Offer

If you are a student currently and looking for cheaper alternatives because “Lakshya we don’t earn yet brother”. Don’t worry kids Azure has you covered, you can try multiple of the services like VMs, SQL Server, Apps, Functions, etc. with a monthly 100$ credit. For our use case just spin up a VM and install all these services individually on it.

If you want to take it a little easy and just run Umbrel instead you can do so by installing docker on your Azure Linux/Windows VM and then run the docker compose service with just a single command.

git clone https://github.com/getumbrel/umbrel docker compose up -d

Mein nah sehta

Enough said already but why am I going through so many steps and things when am well off enough to pay out for all these cloud services. Well the simple answer is

mein na sehta bhai (I can’t bear it brother)

As I already made well known in my previous blog that I hate corporates trying to make me feel like a fool even when am using their services so yes I won’t bow down to them and provide them more of my data to keep in their silos and god knows what they do with it if I ever forfeit to pay for their services I am taking control of my life my data.

Second things come with my upbringing. My cousin Karan taught me to be opportunistic and get the job done without paying a single dime of money. I realize that I am already paying enough of these taxes which don’t provide me back with even basic service, I belong to the most repressed class of humans in Indian Society - A general category, non EWS, salaried, male. So yeah am willing to put in efforts to save a little money here and there.

Thanks for reading this through appreciate it. Looking forward to writing a little more consistently about life, finance, bitcoin and technology of course. You can reach out to me on twitter @1108King

Gold Plated Products is what am calling products/services that get offered as free or for unlimited usage to get numerous people onboarded on it. After a while either they will add a paywall for complete service or some premium features which were previously free are now only available if you pay.

The issue begins when there is a lack of information provided to users of when the paywall or premium offering will become active. The Product Managers just wake one day and decide to choose violence, boom the paywall goes live.

Now user has two options both are miserable:

• Going through the painful process of deboarding from the service through procedures which are never well documented.

• Start paying for the service if it suits your needs.

Google Mail

The original pitch for Gmail was that you wouldn’t have to worry about deleting messages. Instead, you could just archive old emails and find them later by searching.

But in 2013, Google altered the deal, creating a single 15 GB storage pool that covered all of its services, including Gmail and Google Drive. Its free storage limit hasn’t changed since then, and in recent years Gmail users have been feeling the squeeze to become paid users.

Google Photos

When Google Photos launched in 2015 with the promise of free, unlimited photo storage, the pitch was that you’d never have again to worry about photo management. Because of that promise, Google Photos quickly became one of the company’s most popular services, hitting one billion users last year.

But soon enough, Google had a different message for those users who flocked to the service: “Time to pay the piper.”

After using up Google’s 15 GB of free storage, you have to pay for a Google One storage plan starting at Rs. 1,300/year for 100GB.

The product is ad-free, and Google has pledged not to mine users’ photo data for targeted advertising purposes.

Let’s see how long this lasts

Google Drive

Google offered educational institution unlimited google drive space every student in institution fully made use of this as the best years of teenage life does require some space iykyk.

In 2022 they went back on the unlimited storage promise and limited it to 100TB for all the educational institutions. The response they were hoping was for institution to buy more storage but well the administration of most Indian Educational Institution haven’t embraced the need of cloud so they didn’t buy the subscription and are continuing with 100TB only.

Now a student with let’s say around 100GB of video games/movies/photos/videos is now limited to around 20GB (100*1024/5000) and 5GB for alums so they have to move 80GB.

One way is to download everything and then delete other is to copy (not move) it with ease to some another Google Drive. I have tried both and the files you get after download are utter rubbish to parse whereas the internal transfer is made very smooth.

But still you have to delete from original place in both scenarios but if most of your storage is due to Google Photos which was in my case you can’t bulk delete because Google Engineers forgot to add option to delete all photos.

It’s a typical example of vendor lock in 🔐.

So yeah basic quota is 15GB you need 80GB, “Time to pay the piper.”

One Drive

Microsoft also moved away from unlimited OneDrive storage given to educational institution to make them onboard onto the Sharepoint & Teams platform during pandemic. The reason they cited is as below:

Today, most schools (99.96%) are well below their storage allotment. With new tools, all schools can better manage their information and set thoughtful policies for data governance. Some of the benefits we expect to see include helping schools reduce some of the security risks associated with legacy storage and data sprawl, a top concern for most chief risk officers, while also benefiting our shared environmental footprint.

Hoping the my institution falls under the 99.96 category because I moved multiple things from GDrive to OneDrive. But with OneDrive we at least won’t have the trouble of deleting backed up photos as its a folder in your OneDrive which you can delete. So Microsoft at least allows you to deboard unlike Google.

YouTube Music

One of the most recent services released by Google. It’s a very nice service if you have a student ID you get 1 month free, but you have to start auto pay. I tried it, loved the recommendations and while crusing at 170km/hr on Delhi-Vadodra-Mumbai expressway it added even more fun to the journey where even my Mom and Dad started jamming.

It’s been two days since my subscription expired given that I loved it I was planning to get a family subscription (179/month) since it’s cheaper than individual subscription (79/month) if you have a pseudo family of friends also I wanted mom and dad to enjoy it but…..

For these two days they haven’t stopped with the ads before every song you need to waste time on two ads of 5 seconds. Moreover you don’t even have the option to keep music running in the background for scenarios like driving.

Least to say I have been miserable for the past two days and decided not to succumb to this unwanted pressure of subscription hell.

Google Podcast

A very simple un-monetized service to listen to any podcast is being added to the long list of killed by Google.

They are moving podcasts to YouTube music where they can bully us into getting premium.

A finding from reddit:

Got an email today that YouTube Premium Family is going up to $32.99 a month. I was paying $17.99. This and the adblocker crackdown is bloody outrageous. Good luck with the ads kids.

Spotify Music

We all user experience and sound quality that Spotify used to give is unmatched from any other music service. When they started with year in review thing every other organization followed it even Paytm. But now instead of making others follow it they are following Google in making its free users feel miserable. Probably because Youtube Music was snatching away its revenue.

Really I mean the restrictions are quite innovative and absurd:

• prevent manual song selection from a playlist

• disable option to rewind, scrub, or repeat songs.

What’s wrong here?

I got to hear this word for the first time from Archit, user empathy that’s what these product based companies have started missing out on. Why do you follow the policy of lure and capture through vendor lock in instead of making it clear from the start the cost of your service and attached perils of not having using a paid plan.

By giving free users a miserable experience you either end up in two places:

• Get more paid users

• Lose users due to the absolute lack of ethics.

Conclusion

While there is no way to justify this but still let’s understand why it is happening. We already give companies enough data for it earn through it but as ad growth gets tougher, many of them are seeing subscriptions as a big part of their future.

Most of the tax paying + general category + male suffer enough at the hands of the system holding the rant for some other time. So, to tackle this I enjoy being hackish and opportunistic as Bhavya (my mentor) defined this behaviour of mine.

“Bhai mai nah sehta”

To learn more about on dodging the need for all these subscriptions stay tuned on my blog: The Backpacking Dream

What is LFX Mentorship

In the exact words of the Linux Foundation,

The Linux Foundation Mentorship Program is designed to help developers — many of whom are first-time open source contributors — with the necessary skills and resources to learn, experiment, and contribute effectively to open source communities. By participating in a mentorship program, mentees have the opportunity to learn from experienced open source contributors as a segue to get internship and job opportunities upon graduation."

This sounded quite fascinating to me and aligned with my goals. That's how I get all excited for this program.

Why Vitess?

During this term, I had to study Database Management Systems as a part of my college curriculum, and I had previous experience with MySQL, so I applied to the project CNCF - Vitess: Add support for comparing strings using collations and character-sets

Also, I was particularly fascinated by the extent of usage of Vitess. It was built by intelligent people at Youtube to manage their database load and used by Github, Slack, Pinterest, etc.

Problem Statement of Project

Vitess does not yet have support for collations and character sets. So, to compare varchar strings, Vitess needs to rely on WEIGHT_STRING function for now. MySQL documentation says WEIGHT_STRING is a debugging function meant only for internal use.

Use Cases

• Having the ability to compare strings using collation and character-set support, it would be possible to better implement ORDER BY, GROUP BY, JOIN.
• It would be possible to leverage more advanced join techniques than currently implemented.

Plan Out

The initial plan that can be found in the project issue was to use text/collate package. We decided to make its fork and make it suitable for our use case.

We tried our hands to replicate MySQL's WEIGHT_STRING using the package but had no success because:

It only supports UCA compatible collations while MySQL supports several non-UCA collations as well. So in turn Vitess needs to support them too.

So we headed onto the MySQL codebase

Dive into MySQL

Unit Testing Before Anything

The collations implementation in MySQL is part of the MySQL codebase. The first task that my mentor assigned to me was transpiling tests from MySQL to Vitess. It felt weird to me because who writes code first, right?... Guess what I later realized it is a well-known approach called Test-Driven Development.

MySQL had two types of tests, ones that are normal Unit Tests others that are Benchmarking tests. I transpiled both as Unit Tests into Vitess for starters.

Collation Transpilation

MySQL has a lot of different charsets; only a few of them are based on the Unicode Collation Algorithm (UCA). UCA collations are implemented in golang/text. Other collations are custom and need to be ported directly from MySQL.

Extra Collations

conf_to_src is responsible for generating charset data for extra charsets in MySQL. So I was supposed to port this to Vitess.

For parsing the XML Data maps, I learned to use encoding/xml as I was pretty new to golang and its patterns. Vicent Marti, my mentor, helped me a lot by providing me reviews and making me understand go patterns and how to write better, more robust code.

Parsing the XML Charsets was a recursive process where you needed to drop down to levels and merge with previously found charsets their collations.

UCA Tailorings

UCA collations are built on the utf8_0900 collation, where they modify the standard weight as per language needs. Those modifications are called tailorings. Each UCA collation specifies tailorings which then are responsible for modifying weights.

Example: a <<< A << à <<< À < b <<< B

I was to understand the UCA tailoring syntax, copy over the tailorings from MySQL and make a parser to make modifications.

Vicent had a new idea that instead of parsing those strings, we can use the parsing done by MySQL by utilizing the in-memory weights. For that, we decided to dump the UCA weights in JSON using a custom test for each collation.

After that, we diff the ones different from the original UCA weights into separate Go source code that makes our tailorings.

License Issues

Due to Vitess and Oracle license issues, we can't parse the XML as we were doing using conf_to_src. The license only allows us to use in-memory things. So we dump those collations as JSON by attaching to MySQL Unit Testing with a custom unit test and then convert them into go code.

So instead of using encoding/xml, I switched to encoding/json, and we were good to go with a few structural changes. At certain places, I was using regex, which vicent amended to use bufio robust architecture instead.

Result:

Collation Module Integration

After the mysql/collations module was in Vitess we had a meeting with Andres, Harshit, Florent, Vicent and me. In that meeting, we decided on the next step which was integrating the module into Vitess's Evaluation Engine.

Function NullSafeCompare is the one that is used for any form of comparison in Vitess. For Text comparisons is used to return an unsupported type error.

To make it collation aware I added another parameter into its definition that is the collationID. If the ID isn't 0 and we get a collation from collations module lookup function then we use it to collate the values.

Similar to NullSafeCompare we had Min, Max and minmax which were made collation aware so that they can use NullSafeCompare as it needs to be. Unit tests were added similar to ones in MySQL to test the new capabilities of these functions.

So, all the places that were using NullSafeCompare now were supposed to pass down collation information but for starters I just made them pass a 0 and added in TODO.

v1, v2 := "c", "cs"
// hungarian accent insensitive case insensitive collation
max, _ := Max(v1, v2, "utf8mb4_hu_0900_ai_ci")
// max == "cs"

Aggregates and GroupBy

The OrderedAggregate struct was missing collation information it only contained the default charset information in it. To add collation information into semantic analysis Andres and Florent started working on gen4: add collation name to the type definition.

Through PR #4038 Collations were made available throughout the planning phase using semTable's CollationFor(expr sqlparser.Expr) method. Using this method, OrderedAggregate was now aware of its grouping keys' collations.

OA has two methods which call in NullSafeCompare those are:

• keysEqual
• merge

Collations are fetched from OA as a map[int]collations.ID because not all columns have collation info so helps to reduce space requirements.

The map is then passed down to the above functions. If rows with respect to GroupByKeys are equal then they are merged. Aggregation is done based on the opCode for that particular aggregation which in turn might call the new Min and Max.

Finally, unit testing was done for grouping using various collations like case insensitive, accent sensitive and kana sensitive ones.

The following example when grouped using a case and accent insensitive collation:

results in:

Result:

Memory, Route and Merge Sort

Comparer

All these methods are connected by a single struct i.e. engine.comparer. The struct contains details about which column to use for ordering and whether it needs to be descending or not.

It made sense to include collation information too in the struct for ease of access wherever required. Now when calling comparer.compare we can access collation ID and pass it down to the NullSafeCompare call inside the function body.

A new comparer is made from the OrderByParams so they need to carry the collation information as well in form of a new field.

Order By Params

OrderByParams are created using the planbuilder module. To get collation ID we use the semTable.CollationFor(expr sqlparser.Expr).

• For planning Route Sort it already contains details about order abstract.OrderBy. Order contains the expression for the statement on which we can call CollationFor providing us with Collation ID.
• Creating a memory sort plan on aggregation we first need to find the expression that matches our order expression. If we get a match with one of our GroupByKeys then we can access its collation ID for use.
• For memory sort planning with planningContext is similar to route sort we just have to access the semantic table using the context.

Unit Testing

Based on the already present tests for weight string sort with Memory and Route sort testing was done for:

• Ordering by Hungarian Collation
• Descending Order Sorting
• Upper Limit for returned entries
• Error with Unknown Collation
• Error with Unsupported Collation

E2E Testing

Unit testing only tested memory, merge and route sort separately. To test Memory Sort on aggregation we needed to look at E2E tests.

To add collation details to the Order By statement we updated the String() method to add in the COLLATE statement as well with it.

We then updated the aggregation cases to carry collate information in the OrderBy field to be properly able to test the feature. For this, I had to pair code with Andres and Florent because I wasn't much familiar with eval engine testing.

Result:

Learnings

A three-month-long journey and I got to learn so many things:

• Got familiar with golang that too in a project which was one of the early adopters of go.
• I better understand why the design patterns are used in language and can also better implement them.
• Went into a deep dive about string representation in charsets and Unicode which was a prerequisite of the project. Interesting Read
• Got to take a look at the inner workings of MySQL and its humongous codebase and test suite.
• I became collation aware just like Vitess :)

End on High Note

I am very thankful to Vicent for walking me through the mentorship. The reviews I received from him and pair coding under his supervision really helped me grow as a programmer.

Also wanna thank Florent and Andres who were working alongside me on this project and really helped me wherever was needed.

Lastly, I did like to thank my mentor Shivansh Saini for always being a motivation, always will be and one of the reasons I applied for LFX.

Chapel is a programming language designed for productive parallel computing at scale. It simplifies parallel programming through elegant parallel programming constructs. As Chapel can utilize a system to its fullest potential, it seems like one of the ideal solutions to handle millions of network requests.

Currently, we can do network programming by using C Interoperability, but those constructs are not intuitive and good enough for users to easily do their tasks.

My Project

My Google Summer of Code project with Chapel was focused on creating a standard "Socket Library". This project aimed to ensure concurrent, parallel safe, and easy to use constructs for networking.

The project was built on top of C System Calls, saving users from their complexity to perform even the most trivial tasks.

This blog post is for my final submission of the GSoC Project.

• Project: Socket Library
• Mentor: Michael Ferguson, Krishna Dey and Ankush Bhardwaj.
• Organisation: Chapel

Under this project, the following four major parts needed to be managed:

1. Getting review over design for the socket library from other developers.
2. Adding and updating runtime functions that can then be used in Socket Module.
3. Nonblocking but synchronous procedures that allow other tasks to run freely.
4. Creating network constructs for UDP and TCP that align themselves except for the I/O differences.

Pull requests created in the period of GSoC:

Work Done

Design

Even though my GSoC Proposal talked about how the Socket Module should look like its design was entirely based on Python and C System Calls which aren't the best ones and don't align well with how we wanted it to look for Chapel.

My initial task was to look over how other modern-day programming languages implement their Socket Modules. I was recommended to look over at Julia, Go, Python, D, etc. I made did my analysis and brought together a simple gist as suggested to me by my mentors.

Finally, after analyzing implementations of other languages and multiple design discussion sessions with Chapel developers, I made a design pull-request describing the API.

Changes to Runtime

The module that is storing most of the extern procs, records, and constants is the Sys standard module. There were several changes made to the module which were as follows:

• Add in new constants to be used as flags in C System Calls e.g. F_GETFL, SOL_SOCKET to be used in fcntl and setsockopt or getsockopt.
• New opaque types sys_in_addr_t and sys_in6_addr_t which are for standard network addresses like loopback, any, and broadcast.
• Standard Sizes for address and service buffer length to be used in getnameinfo.
• Add in generic getters and setters for sockaddr_storage record.

All this was done as a part of the below PR and soon will be moved to Socket Module when it gets added in.

To treat socket connections like chapel file we needed access to their file descriptor for all the operations for this a minor addition was made in runtime to fetch the file descriptor. This allowed us to use Chapel I/O in the Socket Module which handles buffered I/O and keeps the I/O methods consistent across chapel modules.

Finally, to resolve Domain Name Service from user inputs like "https://summerofcode.withgoogle.com/" I made some fixes to getaddrinfo and constructs related to it whose old implementation had a few issues:

• Some of the function definitions were not correct which were resulting in segmentation fault and memory leak.
• addrinfo wasn't using the new capabilities of chapel runtime and was not interacting with C code it was defined separately.
• As sys_addrinfo_ptr was an opaque type we can't compare it with nil which was necessary for checking the end of the linked list returned by getaddrinfo.

NonBlocking Procedures

The architecture for the socket library was decided to be synchronous but nonblocking as that's how most chapel procedures work ( async is a possibility for the future ). If they need to block they will yield the main thread so that other tasks of the process can run without being blocked.

For this, I decided to use I/O multiplexing using select system calls where the procedure waits until it becomes ready which can mean it becomes readable, writable or a timeout occurred.

The issue was that calling in normal select will cause the pthread on which chapel qthreads are scheduled to block. Hence, to counter this qthreads library provides its own set of system calls that will block but not on the current pthread instead are transferred to another helper pthread where they do blocking.

The outcome was making use of qt_select inside of our sys_select call.

There were certain hurdles and PR remained stale for some time because we faced hurdles due to bugs in error informing inside of qthread's library procedures, which were resolved by collaborating with qthread's core team. The following is the issue that describes the bugs in detail.

Socket Module

The socket module was designed as stated in the design PR and had a lot of modifications being made to the exposed API as reviews were coming in.

The main intent was to hide the complexity associated with the C System Calls under custom procedures and records.

The major records in Socket Module that do all the heavy lifting include

• ipAddr
• tcpConnection
• tcpListener
• udpSocket

Along the lines of Julia utility procedures were provided in the module to user which included bind, naggle, delayAck, getsockname, getpeername, setSockopt and getSockOpt.

The PR has been rebased and changes are being made to it since the finalization of the design for the Module. It consists of all the tests, documentation, and the module itself as well.

Currently, all the tests are passing and PR is under review while performance testing is being done.

Libevent Integration

Libevent exposes APIs that are more performant and handle more open file descriptors or connections than select call ever could. The reason being that libevent uses kernel methods that have a time complexity of O(1) unlike select which is O(N).

Libevent works using callbacks but we have to ensure to make it work in a non-blocking and synchronous manner. Integrating libevent was something that was never done in the chapel so it was a completely new experience for me.

The architecture can be best explained with diagrams, so here goes.

The events are handled by the event loop which runs a callback whenever an event ( read/ write/ timeout ) occurs on the file. It writes back the event's value to a sync variable (passed as an argument to the callback function) transitioning to full state and the procedure waiting on the sync variable is finally woken up.

Usage of Socket Module

A TCP Client in Chapel is as follows:

use Socket;
use Time;

proc main() {
  config const host:string = "chapel-lang.org", port:uint(16) = 80;  
  const x = connect(host, port);
  const writer = x.writer();
  writer.writeln("hello world!");
  writer.close();
  x.close();
}

To serve a basic TCP Server using Socket Module will look as follows:

use Socket;

proc main() {
  var address = ipAddr.create("127.0.0.1", 8000, IPFamily.IPv4);
  var server = listen(address, backlog = 6000);
  writeln("Listening for connections on ", server.addr.host, ":", server.addr.port);
  defer server.close();

  while (true) {
    var conn = server.accept();
    const address = getpeername(conn);
    const writer = conn.writer();
    writer.writeln(address, " connected ");
    writer.close();
    conn.close();
  }
}
`

We can utilize chapel begin to demonstrate UDP connections in a single program.

begin creates a new task or user-level thread which has a separate execution context from the main task/thread providing us capabilities for task parallelization.

use Socket;

proc main() throws {
  var address = ipAddr.ipv4(IPv4Localhost, 8111);
  var receiver = new udpSocket();
  bind(receiver, address);

  var sender = new udpSocket();

  var sent = b"hello world!";
  begin {
    var sentBytes = sender.send(sent, address);
  }
  var received:bytes;
  received = receiver.recv(sent.size);
  writeln(received);
}

Weekly Updates

Updates for all the weekly meetings that took place can be found on this notion site.

Future Work

• Integrating libevent with the chapel channels through an I/O plugin.
• Running performance tests to analyze the network calls and I/O latency.
• Supporting opaque types with, the LLVM backend.
• Providing users with libevent as a dependency if they don't have it on their system.

Learning

• The major learning outcome from this project was good coding practice. I learned how to write DRY, reusable code following SOLID principles. I also learned how to write good tests/specs covering all possible scenarios.
• When writing the Design for the project, I had long discussions with my mentors trying to decide on exposed APIs. Coming up with good names for methods, variables, and hiding the complexity from end-users.
• Through this journey, I learned a lot about how open source projects work, getting involved with the community, and contributing to the projects.
• The major tech takeaway for me was how familiar I got in working with C System Calls, Pointer Exceptions, and Chapel itself.
• I am still exploring Asynchronous I/O through libevent for the project which is something that powers fast servers like NodeJS, Julia, and Go.

Special Thanks

It was wonderful working with the Chapel community throughout the GSoC project. I would like to thank Michael, Krishna, and Ankush for solving my queries, debugging issues with me, giving constructive suggestions and guiding me through the program. Thanks to Brad, Lydia, Engin, Greg, Paul, and the members of the Chapel Community for helping with their suggestions.

I would like to thank Shivansh Saini, Nishant Mittal, and Ashish Kumar for motivating and guiding me in applying and getting through the whole program.

Final thanks to Google for organizing this amazing program. I feel GSoC made it easier for me to get started with open source contributions. It was a great lesson on how projects are built and maintained with great quality. I am excited to continue contributing to the open-source community

Thanks for reading :) and please share your thoughts. I always like to meet more enthusiastic people, so do reach out to me on Twitter or Linkedin to share your feedback or for any queries. I'd be more than happy to help and connect.

Why won't it be its the company which develops typescript making web app development so much easier, VSCode favourite IDE of all developers 😌, Azure Cloud Services, Windows the first Operating System that most people use.

The Process Included two parts the Coding Round and then Interviews, based on which students were selected. While Coding Round was eliminating Interviews were not, so everyone was able to give 2+ interviews.

Coding Round

Microsoft provided us with two questions on Codility. The first one was an easy greedy + sorting question where you have to check by using available cranes can an object be moved from B to E. Sort the (crane position - length , crane position + length) then keep on moving B till it reaches a crane that has B and E in the same range. Made sure to swap B if it's greater than E. That was that.

Another one was also an easy greedy problem. We were given that a player made T dice moves off which he only remembers N moves exactly and mean of all the dice moves. So we just have to find the remaining possible value of move which can be done very easily by dividing the remaining sum ( found using mean*T - the sum of N) among all values equally and if some still remain divide it one to each.

The thing about Codility is they won't run any test cases in front of you they are all hidden which I only realized after submitting 1st question.

Interviews

2 Rounds were conducted for me there was no clear separation as to whether they were HR or Technical. I was asked HR-related and technical questions in both rounds.

Round 1

Introduction

The interviewer asked me about myself. I had been finding the right answer for this for quite some time now. I started with personal info, then what I do? which is open source contributions, technical blogging and attending conferences. He seemed impressed as he told me that wasn't the usual answer he gets.

I also told him about how I started my technical journey as a child curious to get acquainted with Windows XP and MS Office. Also, my current stack is Visual Studio Code and Typescript, both developed by MS again 😄.

Knight and Destination

The was that question given a n*m chessboard, and you have a Knight piece that needs to go from S to D, find the minimum number of moves required to reach destination D. I started with a crude simple BFS solution, and before coding, I explained my approach to him. After a go-ahead, I coded the initial solution, which was un-optimized. He asked me to do a quick dry run to show that it works.

Next, he asked me how would I do memoization so that the complexity is not exponential. So I implemented memorization using a n*m array which stores the minimum move from that location (x,y). Next, he asked me how can I do pruning for the solution so that we don't go to longer paths if we already have a minimum implemented that was just an if-else check on the current minimum to stop when needed.

Finally, he asked me, since it was uninformed BFS, what better could be done. I told him that Dijkstra's Algorithm would do the job nicely, but since it's a matrix, we can use A* Heuristic Search Algorithm to do even better. I showed him why this would happen by defining the heuristic function as manhattan distance. I wasn't asked to code them, just what and how better?

Why Microsoft?

Finally, he seemed satisfied and asked me why Microsoft I also had it covered. My answer was that I wanted to empower new developers to build even better software using services like Azure, Typescript and VSCode which will give me a sense of pride that the software I built is being used by developers to create even better services for users.

Anything Else?

He then asked me if I wanted to ask anything. I asked him what he does, and day to day workings of his at Microsoft. He gladly told me about the project he was working on which included created higher-level APIs for Database manipulation in Azure.

Round 2

Level Order Traversal Binary Tree

This started without my introduction. This time he gave me a problem after introducing himself. The problem was printing level order traversal of a binary time. This time I was asked to code first then explain, so I quickly went ahead and gave a solution using queue and yeah, it seemed good to him.

Although he stopped me midway when I used auto in C++ because that surely makes it difficult for humans at the end, humans should understand code, not just machines. Finally, I did a Dry Run of the implementation and was done with the question.

Explain Any Project

Next, he asked me what was the most difficult part of any project I had mentioned in my resume. I asked him do open-source contributions count? and he was impressed and went on to tell me that at Microsoft, they value Open Source so yes they count. So I told him, in brief, all the issues I had when I started with DXHeroes and PGRouting also talked a bit about my GSOC project, which I didn't particularly tell as GSOC project but just any open source contribution.

Talk about GSOC Project and Async I/O

As open source is huge, he wanted to know just something specific he asked me about my GSOC Project at Chapel, which I cited as a productive parallel programming language. I told him that this was my first time working with C System Call, joked about getting my hands dirty with pointers, allocation, etc. ( he laughed too xD ).

I finally told him about issues with I/O multiplexing, which I was resolving using Asynchronous I/O. Async IO hit him as something that college don't have experience with, and he himself worked on it, so he told me, like, yeah, that is a totally different game.

Anything else?

Finally, my chance to ask, I asked him what interns do whether their code makes into actual user apps or not.

End

Some people had round 3 as HR, or Technical mine wasn't, so after the interview, I was sad, but Training Placement Cell Representative told me that it wasn't the case that I was eliminated, so was happy again.

And Yeah Selected 😊

Thanks for reading :) and please share your thoughts. I always like to meet more enthusiastic people, so do reach out to me on Twitter or Linkedin to share your feedback or for any queries. I'd be more than happy to help and connect.

Google Summer of Code is a global program focused on bringing more student developers into open source software development. Students work with an open-source organization on a 10-week programming project during their break from school.

This post highlights my journey of getting selected into the program from the very beginning. I believe that there is no defined path for getting selected in such open-source fellowships. Everyone has a unique journey, and you can take a few things from other journeys to write yours :) because the essence of each one is similar.

Now that I have your attention, let's get started. I will talk about my journey in two parts:

• the initial Pre-GSoC part will cover how I came about making my first contribution in Open Source and developed my skills
• the later part GSoC InFocus focuses on GSoC specific things like selecting an organization, contributing, writing a proposal, etc.

Pre-GSoC

This was a phase where I wasn't much worried about GSoC and focused on developing my skills, adding new tech stacks into my belt, developing more projects, and contributing to open source.

This phase is crucial because your zeal and fire to contribute and make a mark in the open-source community starts to rage, and doing some fuels that fire further.

How I came to know about GSoC?

The first time I heard about GSoC was in an event conducted by Club of Programmers IITBHU where I was introduced to open-source programs like GSoC, Outreachy, etc. and people who were previously selected shared their experience with us.

I wasn't sure whether I would someday dwell into the realm of Open Source as, at that time, I was more focused on Competitive Programming like most of my peers. But things changed after the programming Camp in the Winter of 2019, where I started with Web Dev, and now I was sure I want to be amongst the group of people who clear GSoC.

Then came the GSoC'20. Many of my seniors who applied got selected, and I thought that a first-year student could never clear GSoC ( that perception changed this year :) ), so I focused on improving my skills in Web Development and didn't bother applying in 2020.

Learning Phase

I was burning the midnight oil to make web development my forte be it backend or frontend, vue or react, Django or NodeJS... I wanted to know it all as there existed so many fascinating things for me to learn.

So here are some of the things I did to strengthen my skills in web development :

• I utilized Frontend Masters to cover most front-end things like frameworks and Design. Also took a git course by @nnja
• For Django, I started dwelling into the docs of Django and DRF to learn more advanced features after completing basic stuff from DjangoGirls
• Then, I did some projects using what I have learned to combine both backend and frontend. I did invest quite some time in them, and it was fun.
• Later on, when I came to know about things like NodeJS being faster and a more scalable DB such as MongoDB, I did a project using them in a Hackathon with my friends @Purushottam and @Madhava, which we won too 🥳

Apart from my own projects, I participated in projects of our Programming Club and also did an intern in a freelance project. These things prepared me to swiftly walk through a large codebase, collaborate on Github, use version control, apply my knowledge, etc.

Open Source Hacktoberfest

There couldn't be a better start for your open-source journey than HacktoberFest. It's an annual open-source event where project maintainers await your contributions, and they prepare their projects to accept new contributors.

Apart from contributing, you also get rewarded swags, tees, etc., by DigitalOcean and some of the projects. You can check out hacktoberfestswaglist during hacktoberfest to know about projects offering additional swags.

I outlined my hacktoberfest with my friend Kushal ahead of October. We decided to go good-first hunting, work on several issues and earn some swags. Who doesn't like swags, right? So I intended on completing the tasks to get those swags.

All of the projects were very welcoming. Even though I had no idea about the tech stack they were using, I contributed to them, but I focused on solving issues and asked many questions.

Some of my contributions which you can check out :

• DX Heroes
• Operation Code
• Datenanfragen
• Letra

By the end of October, I was familiar with contributing to open source projects, interacting with communities, finding issues, making PRs, and working my way around good first issues.

GSoC InFocus

For most people, this phase is after and around December and January, where you start looking at organizations finding relevant and interesting projects to contribute to.

Selecting Organizations

I started looking at GSoC archives in January and probably would have gone through the whole A-Z list about 10-20 times. Earlier, they didn't have filters for the tech stack, but they have it now, and it really helps to select the organization.

First, I jotted down the list of organizations I want to contribute to based primarily on my interest in their past and current projects and those that didn't require me to learn something completely new ( biggest mistake '-' ).

One of the most useful tools I found when selecting my GSoC organization was developed by one of my seniors Nishant Mittal. With all the statistics at hand, I can filter out organizations.

The list, as you can see, consisted of around 30-40 organizations. So I started filtering them out based on:

• Number of times org was selected for GSoC in past
• The tech stack they were working on should align with what I know, things like Javascript, Python, NodeJS, Rest APIs, etc.
• Number of students they select, which I filtered to be at least greater than or equal to 4.
• Amount of people already contributing to them because eventually only a few students will be selected for summer.
• Organizations using IRC as a means of communication were a big NO! for me.

In the end, I decided to go with HTTP-APIs. I started checking out their Contributing Guidelines, Resources to get started, and their repositories. I selected HTTP-APIs because REST APIs are my forte, and they were doing something pretty interesting.

I made a few contributions to their repos, upgraded their dependencies to support python v3.8 and updated Dockerfile.

Still, a few things that took me away from them were that the mentors weren't that responsive, and even though I went through all of their resources, I couldn't find something significant to contribute.

So I drifted away from HTTP-APIs, but at the back of my mind, it was still that I have to make it work there somehow.

Change of Plans

If you know me prior, you already know that I never went through with HTTP-APIs instead, it was Chapel. Chapel is a programming language designed for productive parallel computing at scale.

The story behind the finding of Chapel is something that you don't hear often. I was in a meeting with my peers, and we started talking about things we are doing, and GSoC became a part of the discussion.

My friend @Jatin stated to me that, and I quote, "there is this language where you can run statements simultaneously using a forall loop". He seemed pretty interested, and so sparked the idea of taking a look at Chapel.

It was also at the same time, i.e. February, I was talking with one of my seniors @Ashish, who previously got selected in GSoC under OSGeo. My primary concern was that HTTP-APIs didn't work, and if I go in places related to Web Development there, the amount of already active students is way more than I wish to compete with.

So the plan I made after discussing with Ashish was to contribute to something, not JS, not Python, but things constituting C++, C, or some new language. And since Chapel caught my interest, I started there.

It is hard to find a project where you have experience with all the technical tools, so in hindsight, it’s okay for you to select an organization where you have only 40% of the tech stack down. You might even end up working with the part of the software that uses a language you have zero familiarity with, like me.

Contributing to the project

The thing with all GSoC organizations is that they have excellent resources for you to get started with. This includes documentation, contributing guidelines, getting started with contributions, etc.

Just as HTTP-APIs, Chapel also had excellent resources for me to learn it, and I went straight for them started writing small scripts, learning the syntax incrementally.

The question you might have for me is why I decided to stick around with Chapel? The difference was chapel has so many issues ( over 2k ), and these are the ones made by the core team itself. That made it pretty clear that I can contribute a lot and the team is highly active in developing the project.

I was observant even when reading the documentation and talked in the gitter channel about how the documentation can be improved. I was also helping out by answering some of the questions in the channel.

Something that usually people forget about is interacting and being a part of the community. It's not just about coding your way around, and I made sure that I opened gitter every day to take a look at the ongoing discussion here as well as on Github Issues.

As chapel is a comparatively new language, I knew there could be scope for improvement on the implementation done in Modules. Also, at the time when I started contributing, I was pursuing courses on Algorithms and Operating System as part of my college curriculum.

So I went on to find the implementation of Quick Sort and Heap in the codebase, discussed few updates on them, and made a Pull Request.

After that, I started looking at existing issues, discussing how I should work them out, the idea of the core team about the issue, and eventually made the Pull Request for it to be reviewed.

The thing that kept me engaged in chapel was the quick response from mentors within a day or two. That made me feel that my contributions are very much appreciated.

Writing Proposal

It is expected for a proposal to contain a detailed explanation of your implementation and timeline. What gives you an edge is your contributions to the project until then. This helps solidify the trust in your abilities to finish the project.

Even though I had drifted away from Web Dev, it all came back when projects that were announced by Chapel had a project on Socket Library. This was something that I was targeting since it was announced even before organizations were announced.

Even though Chapel gave me a place to contribute to it was the project that kept me going because it's my niche and was something where I can apply all the knowledge I gathered over time.

I started working on my proposal after getting clarity on few things from my mentor on what were they expecting from the project. I also used to look out for questions that other people were asking related to my project.

I got up to speed on working with sockets in C, went through the design of the socket library in Python, gathered more information about non-blocking sockets, and finally designed my proposal.

I made sure to add in code snippets and mock implementation of how the code will be implemented in C and Chapel. It is also a good idea to add in diagrams which I did, because images speak more than text.

My organization mentors seemed happy with the design because they didn't seem to have much issue with what I had shown them in the draft.

But @Shivansh, one of my seniors I requested to review my proposal, told me that it was less technical and needed more of it. This pushed me to improve my proposal further, add more details about the implementation, add in diagrams and code snippets.

You can find mine and several other accepted proposals in this repository. Courtesy of Club of Programmers IIT BHU

Post Proposal Submission

Going inactive after proposal submission shows that you aren't interested in the community, and you were doing it just for namesake.

I ensured that after proposal submission, I kept contributing and was active not that much but still had my presence in the channel. I wrote in my proposal that I would get all my pending PRs merged, and that's what I did.

Fast forward to 17th May, I was in a voice call when the result came out, and I literally shouted woohoo!

Final thoughts

GSoC is a non-competitive (after getting selected, of course 😏) program so try to cherish the community alongside it. Find your motivation; why do you even want to do it first, and the rest will follow.

GSoC is not an internship and should not be looked at as a way to earn money. It is definitely not something you should pursue half-heartedly or just for the sake of it, neither it would help you nor the community, and there can be better ways for you than open-source.

I am learning a lot from my mentors and fellow GSoC mentees. The community is something I will definitely enjoy and use my summer to learn from them and, as I say, make Chapel go to Web.

Thanks for reading :) and please share your thoughts. I always like to meet more enthusiastic people, so do reach out to me on Twitter or Linkedin to share your feedback or for any queries. I'd be more than happy to help and connect.

When writing Dynamic UI Components that respond to user actions and events, we require methods to respond to these events by translating them into our DOM. Change in DOM is usually achieved by changing styles and classes based on certain reactive data that we have in our javascript.

While we can certainly do string concatenation, calculate a string and then bind a string to class or style... this method is error-prone and cumbersome at times to deal with. That's where Vue.js's clean suite of enhancements come into vue ( french for 'view' )

Quick Recap Data Binding

If you aren't familiar with what data binding is....it's essentially binding any attribute of an element in your template to the data available in your script tag, which can be props, data or computed properties.

Data binding is one of the most elegant features of Vue.js because it provides reactive data binding with a straightforward syntax using v-bind.

<div 
  v-bind:class="dynamicClass"
>Hello World!
</div>

A shorthand for data binding is but just using : and then the attribute name, which I guess anyone would prefer using more.

<div 
  :class="dynamicClass"
>Hello World!
</div>

Let's suppose that the above class is not a once initialized and stays the same kind of class it changes based on user input, so we have to use a computed property or watch to make changes to our dynamicClass variable. So things will start to look like this.

export default {
  data( ) {
    return {
      changingBoolean: false
    }
  },
  computed: {
    dynamicClass: ( ) => changingBoolean : 'text-center text-lg' ? 'text-justify text-xl'
  }
}

Looking at the code above we can see that for a simple switch in classes based on a variable we had to write so much code. So simply creating a dynamic class won't work.

Array Syntax for Classes

Enter array syntax which makes the previous task less cumbersome and also keeps it DRY at times when needed.

<article 
  :class="[changingBoolean : ? 'text-center' : 'text-justify']"
>
    Hello World!
</aside>

This looks so much cleaner than the previous method right ≧◠‿◠≦✌. But it's an array so we can add multiple values into it too :). Now we can toggle the text alignment class while flex and width will always be present.

<article 
  :class="[changingBoolean : ? 'text-center' : 'text-justify', 'flex w-2']"
>
    Hello World!
</aside>

Object Syntax for Classes

Sometimes we just want to add toggle a single class on/off when a boolean is true and nothing when it's false. Using ternary operator it will look as below

:class = [changingBoolean : ? 'text-center' : ' ', 'flex w-2']

We can do better, enter object syntax because eventually, everything is an object in javascript so why not.

:class = [ { 'text-center' : changingBoolean }, 'flex w-2']

You can also bind an object directly to class instead of keeping it inside an array and it also supports multiple togglable classes just like an array.

<article
  class="absolute"
  :class="{ active: isActive, 'text-xl': largeText }"
></article>

active is a simple string variable for class whereas isActive and largeText are boolean variables. Also if you noticed class and :class can simultaneously exist on a single element ツ

Passing in Objects

We can also pass in reactive array/object stored in our data or computed to classes. This can be a more powerful pattern at times when you have to do multiple checks and toggling which when accommodated into HTML won't look good and readable.

<nav :class="classObject"></nav>

Modifying Child Classes

Suppose we have a nice and shiny icon element we have specified several classes to it which works for most cases so we didn't bother making it a prop. But a time came when we had to change its colour in that case we want to pass down a new class to our child.

<my-icon
  :class="text-blue-600"
/>

Now the :class will be appended at the end of the class inside of our component's parent. We can obviously also send in a simple _class_ too.

Array and Object Syntax for Styles

The array and object syntax for classes and style looks exactly identical except for a very minor change. It's not about the truthiness of variables anymore it's about assigning them to the right CSS property.

<nav 
:style="{ marginTop: marginTop + 'px', backgroundColor: infoColor }"
>Doge Coin
</nav>

In the above example, we are assigning the color property a dynamic value and a similar operation for fontSize.

• We can write properties as kebab case too just ensure to wrap them in quotes
• It can be more powerful to directly pass in an object to style which is a more readable and cleaner method.

The purpose for array syntax in style reduces to allowing us to pass in multiple objects ( Duhhh that's what arrays do right :P ) for style as passing a string to style works won't make much sense in the special syntax.

<nav 
:style="[marginObject, backgroundObject]"
>Doge Coin
</nav>

Bonus Treats

Some CSS properties require us to use vendor prefixes. Vue will apply them for us implicitly but if you want to be explicit you can pass in multiple values for a single property through object syntax and providing an array of values. Vue will only render the last value in the array which the browser supports.

<ul :style="{ display: ['-webkit-box', '-ms-flexbox', 'flex'] }"></ul>

Thanks for reading! :). Please share your thoughts about the array and object syntax would you prefer them over strings?

Reach out to me on Twitter to share your feedback or for any queries. I'd be more than happy to help!

Regular expressions ( or regexes, or regex patterns) are essentially a kind of formal grammar/syntax used to find the set of possible strings that you want to match.

At first, REs can look pretty scary and daunting, but we can write highly efficient matching patterns in terms of length and speed after understanding even a few special characters.

We can use REs to save the day in a variety of use cases :

• Checking a pattern in string
• Splitting into strings
• Replacing part of strings
• Matching a complete string
• Cleaning raw data in Natural Language Processing

All major programming languages and even IDEs have their own standard module for regex where the syntax might change, but the underlying concept remains the same.

# python3
import re

pattern = re.compile("doge\\shen+lo+",re.I)
if pattern.match("Doge hennnloo") is not None:
  print("Henlo Doge")

// javascript
const pattern = /doge\shen+lo+/i
if (pattern.test("Doge hennnloo")) {
  console.log("Henlo Doge")
}

Let's get started 🚀

Normal Characters 🇦

You can use regex as you use normal strings/characters for matching too :

console.debug(/king-11/.test('king-11')) // returns true

But certain special characters have to be escaped because they carry special meaning in regex.

\ / [ ] { } . ? + * ^ $ | ( )

// we have to escape back slash in string to be tested as well
console.debug(/\\\/\[\]\{\}\.\?\+\*\^\$\|\(\)/.test('\\/[]{}.?+*^$|()'))
// returns true

Character Class and Regex Characters 🧑🏼‍🤝‍🧑🏼

A character class allows you to define a set of characters from which a match is considered if any of the characters match.

/[aeiou]/.test('e') // returns true
/[aeiou]/.test('c') // returns false

You can also provide range values to regex for defining character classes using -.

/[a-z]/.test('l') // returns true matches all lowercase alphabets
/[A-Z]/.test('L') // returns true matches all uppercase alphabets
/[0-9]/.test('8') // returns true matches all digits from 0 to 9
/[0-5]/.test('8') // returns false matches all digits from 0 to 5

But if you want to match -, you have to escape it or keep it at the start or end of character class, i.e. [].

/[0\-5]/.test('-') 
// returns true matches 0, 5, -
/[-0-5]/.test('-') 
// returns true matches digits from 0 to 5 and -

We can define complement character class, i.e. characters we don't want to match, by adding ^ at the start of our class.

/[^a-z]/.test('A') 
// returns true doesn't match any lowercase alphabet

Inside a character class, only two characters carry special meaning - in-between characters and ^ at the start of the class rest all other characters don't carry their special meaning; hence we don't need to escape them.

/[$+]/.test('$') // returns true matches $ and +

You can define character classes for things like alphabets and digits but regex makes it easier for you by defining several special regex characters:

• \w [a-zA-Z0-9_] Alphabets and Digits Class
• \W [^a-zA-Z0-9_] Negated Class of \w
• \d [0-9] Digits Class
• \D [^0-9] Negated Class of \d
• \t Tab Character
• \n Newline Character
• \s [ \t\r\f\v\n] Matches all white space characters like space, tab, newline, carriage return, vertical tab, form feed, etc.
• \S [^\s]
• \b Matches Word Boundary where there is a \w on one side and \W on other side of position
• \B [^\b] Matches all non Word Boundaries

Wild 🐯 Cards

• ^ symbol allows us to match the starting of the string
• $ allows you to match the end of the string
• . allows us to match any character

console.log("Tall".match(/l$/ig))
// [ 'l' ]
console.log("Tight".match(/^t/ig))
// [ 'T' ]

In the above example, while l was matched only with the last one, whereas T was matched with the first one due to $ and ^, respectively. Without $ as you can see, all the ls were matched.

regex = re.compile("ho.dog")

print(regex.search("hotdog") is not None)
# True
print(regex.search("hoddog") is not None)
# True

. allowed us to match both d and t in the above example. All the three wildcards are special characters to match them specifically, i.e. ^, $ and . you have to escape them using \.

Flags 🎌

You might have observed usage of g or i after the regex expression so what are these things? These are regex flags that affect the number and type of match we want to make :

• i - Case insensitive match which doesn't differentiate t and T
• g - Global Flag allows us to match more than one instance of our expression and not just the first instance
• m - Multiline Flag affects the behaviour of ^ and $ where a \n newline character would mean the start of a new string

  import re
  print(re.search("^football","rugby\nfootball",re.I|re.M))
  # <re.Match object; span=(6, 14), match='football'>
  print(re.search("^football","rugby\nfootball",re.I))
  # None
  

• s - DotAll Mode, allows wildcard . to match newline character as well.
• u - Unicode support enabled

Half Way through, but I believe the next half will be easier

Quantifiers 3️⃣

Sometimes we require to match a character class, character, group ( coming up ) zero, one, more than one or even, let's say 11-22 times ( random numbers ); in such cases, quantifiers come to the rescue:

• ? Matches its preceding character, class or group zero or one time.

  regex = re.compile("hot?dog")
  
  print(regex.search("hotdog") is not None)
  # True
  print(regex.search("hodog") is not None)
  # True
  

• * Matches its preceding character, class or group zero or more times ( ∞ ).

  regex = re.compile("hot*dog")
  
  print(regex.search("hotttttdog") is not None)
  # True
  print(regex.search("hodog") is not None)
  # True
  print(regex.search("hotog") is not None)
  # True
  

• + Matches its preceding character, class or group one or more times ( ∞ ).

  regex = re.compile("hot+dog")
  
  print(regex.search("hotttttdog") is not None)
  # True
  print(regex.search("hodog") is not None)
  # False
  print(regex.search("hotog") is not None)
  # True
  

• {n,m} Matches its preceding character at least n times and at most m times. The default value for n is 0 and the default for m is ∞

  regex = re.compile("hot{1,3}dog")
  
  print(regex.search("hotdog") is not None)
  # True
  print(regex.search("hottttdog") is not None)
  # False
  print(regex.search("hotttog") is not None)
  # True
  

Groups ✨

Groups allow us to create grouped expressions that can help us in substitution, referencing them in later parts of a regular expression.

Let's say we want to replace all the function with arrow functions. Obviously, we would like to retain the name of the function and its arguments, but we need to reference them in replacement. With VSCode, our favourite editor and regex, we can do something like :

function NotAnArrow(argument) {
  console.log("I do something")
}

const NotAnArrow = (argument) => {
  console.log("I do something")
}

What we used were capturing groups, which we created using (.+) and (.*) ( arguments might not be there ). Anything inside those brackets forms our group, and the expression inside them is the one that will be matched.

There are other types of groups as well:

• Named Groups (?<Name>...) is a group that provides us reference to that group using its name instead of numbers
• Non Capturing Groups (?:...) is a group that will match, but we can't reference them in the result of the regex expression.

Alternation |

Alternation is a simple OR that we can use between different parts of our regex pattern to match this or that or even that by providing different options, just like radio buttons.

const regex = /(java|type)(?:script)?|html5?|css3?|php|c(\+\+)?\s/gi

for (let x of `JavaScript is here but PhP came
before them and now its TypeScript`.matchAll(regex)) {
  console.log(x)
}
// [ 'TypeScript', 'Type', index: 56, input: 'JavaScript is ..... ]
// .
// .

So here is what we did with that complex-looking but now easy for you to understand Regular Expression.

• Created Group to match both Java, Javascript and Typescript using | and ?
• Used | to match other languages as well
• Escaped + to match for C++ and C as well using ?
• Finally, a character class \s to signify the end of the language name
• Flags like gi to match all languages and irrespective of case

Usually, you will find yourself using | inside of groups as you wouldn't want to pollute your global regex just like you always use virtual environment. Make sure to make them non capturing if you don't want to reference or find them in your results.

Trick 🎃 Treat

+ and * are of nature greedy. They will keep adding characters to the match until they find the last instance of any succeeding expression or the sentence ends.

import re
regex = re.compile("(a|m).+e")
print(regex.match("apple maple"))
# expected ['apple','maple'] found 'apple maple'

Here .+ ignored the e of apple and went all the way to finish at e of maple as it was the last instance of e that it was able to find.

Lazy Mode for * and + can be activated by appending ? after the quantifier matches only the minimum required characters for the regex expression.

const regex = /(?:a|m).+?e/gi

for (let x of "apple mapple".matchAll(regex)) {
  console.log(x);
}
// expected ['apple','maple'] found ['apple' 'maple']

I highly recommend you to check language-specific docs for syntax and available features. One of the best docs for regex is for python.

I didn't cover one feature that I might make a new article on it is Lookahead and Lookbehind. You can wait :) or check out the link.

Why did I say that? Because sometimes regex can take exponential time to search a be really catastrophic. So .....

May the code be with you 🖖

Why bother?

• Writing better commits can make the difference between your debugging session lasting 1 hour to a few minutes.

• With good commits, it's easier for anyone to understand the progress of a project and get up to speed.

• It also allows reviewers to understand your progress during the resolution of an issue clearly.

• Generating good commits will allow you to track back changes/issues to a specific module/file and associate it to the code easily.

• They enable you to understand how the issue crept in the codebase and how you can easily resolve it.

• Good commit messages can save you the hassle while creating release notes

What are good commits?

Modular

A commit should be modular in nature, just like our codebase. Each commit should encapsulate a single logical idea only that might span over several changes but as a whole is a single related group of changes.

E.g., add CSS to a component, change the margin of all pages, strong typing your functions, etc.

Type of Commit

Categorising your commits into types also clarifies what type of change it made and can be identified instantly. I usually keep a template file that contains those types so I don't have to remember them :P

Some of the subtypes can be a chore, feat, fix, refactor, style, etc. What meaning does each carry can vary based on your team or you as an individual as well. Below are the types that I use in my commits :

• feat: new feature
• fix: bug fix
• refactor: refactoring production code
• style: formatting, missing semicolons, etc.; no code change
• docs: changes to documentation
• test: adding or refactoring tests no production code change
• chore: updating dependencies etc. no production code change

Description

A commit message except for the subject line limited to 50 chars should include a description broken into 72 char lines/line. The description is separated from the subject by a blank line. The description should try to cover things like :

• the current behaviour,
• what change are you making?
• why this fix is needed?
• why take the current approach for this change?

Commit message should be in the future tense, i.e. 'Fix' vs 'Fixed.'

Bonus Treats

You can specify a template for your git commits by adding the following config to your $HOME/.gitconfig

[commit]
    template = ~/path/to/template

My template looks like this, which allows constraining the number of characters in the subject and description line. I also add the types and just in case I don't remember :'(

# <type>: <subject>
##### Subject 50 characters ################# -> |


# Body Message
######## Body 72 characters ####################################### -> |

# --- COMMIT END ---
# Type can be
#   feat    : new feature
#   fix     : bug fix
#   refactor: refactoring production code
#   style   : formatting, missing semi colons, etc; no code change
#   docs    : changes to documentation
#   test    : adding or refactoring tests
#             no production code change
#   chore   : updating grunt tasks etc
#             no production code change

Note : For your template to be prompted, you have to use git commit.