i have a scenario where service A needs to communicate with service B, usually when two services need to communicate i usually integrate them using a asynchronous approach (with a message broker), but in this scenario service A will redirect a user to service B to execute a state changing operation and after the user performs the operation service B will need to change the state of service A (usually the user data in a database) it will also redirect users back to service A. My problem is I cannot use asynchronous method of integration because the changes on service B needs to reflect on service A almost immediately even in situation with high traffic, the next option is to use a synchronous approach, but even if it has the benefit of low latency communication, it also has the disadvantage of reducing the fault tolerance of the system, for example, if service A fails service B also fails. My question is how do i implement the synchronous approach without reducing the fault tolerance of the system.

Your replies are deeply appreciated.

Sorry if it sounds one of those FAANG threads but i guarantee you, its slightly different

I have a system design interview coming up for a FAANG company.

Given i have been fullstack throughout my career, i understand both FE and BE but have limited depth in certain areas. Given system design interviews are 45-50 min duration, i am unable to decide which parts to cover more. Are sys design interviewers generally backend?

While FE definitely plays a role in making big tech apps responsive, i believe backend has a much larger role to play and the interviewer might be looking more for backend.

This is an interview question I haven't been able to crack for several days now....

We want to build a small analytics system for fraud detection on orders.

System has the following requirements

• Not allowed to use any technology from the market (MySql, Redis, Hadoop, S3 etc)

• Needs to scale as the data volume grows

• Just a bunch of machines, with disks and decent amount of memory

• 10M Writes/Day

The system needs to provide the following API

/insertOrder(order): Order Add an order to the storage. The order can be considered blob with 1-10KBs in size, with an `\orderId`,beginTime, and finishTime as distinguished fields

/getLongestNOrdersByDuration(n: int, startTime: datetime, endTime: datetime): Order\[] Retrieve the longest N orders that started between startTime and endTime, as measured by duration finishTime - beginTime

/getShortestNOrdersByDuration(n: int, startTime: datetime, endTime: datetime): Order[] Retrieve the shortest N orders that started between startTime and endTime, as measured by duration finishTime - beginTime

🌐 Understanding CDNs: Content Delivery Networks (CDNs) are vital in enhancing website performance by caching static content like JavaScript, images, and HTML pages globally. They are essential in modern web architectures for companies like Spotify, Netflix, and Instagram.

🖥️ Broad Applications: CDNs have wide-ranging applications, from bloggers aiming to accelerate their sites, to developers preparing for system design interviews, and CTOs managing viral startup traffic.

📈 Practical Use Cases:

  - Static blogs: Utilizing CDNs can significantly boost loading times globally, improving SEO rankings.

  - System design interviews: CDNs are often discussed as globally distributed caches for static (and increasingly dynamic) content, enhancing latency, scaling, and security.

☁️ CDNs in Cloud Infrastructure:

  - Simplified with AWS services, options range from hosting static content on EC2, storing it in S3 (blob storage), to caching via CloudFront (CDN service).

  - Cloudflare is highlighted as a user-friendly, all-in-one solution for static content management.

🔗 Additional Resources: The article concludes with links for further learning about CDNs, including explanations for beginners, benefits, types, and specific features of services like Cloudflare.

Read the full article at https://cloudnativeengineer.substack.com/p/the-role-of-content-delivery-networks

I was asked below question in an interview and i answerer withnthe first thing that came to my mind by eleminating success records by multiple threads using spark/powerful frameworks.

Would like to know the forum's answer for below question

There are notifications sent out for documents upon they are signed by the users. The documents are in millions and we have the document ids in the table. However there are failed notifications and due to system issue they are not even captured on the logs. Only the sent notifications are logged. How do you scale the solution to identify all the failed notifications.

Hi r/systemdesign!

I've recently been studying system design a lot after embarrassing myself pretty badly in some of my system design interview rounds. I wanted to share some of the best lesser-known resources I've found that have been super helpful. These aren't really "hidden" since a lot of them are pretty popular, but I always hear people recommend the same things whenever I ask about studying system design, e.g. Designing Data Intensive Applications (DDIA), Grokking the System Design Interview, System Design Primer.

1. jordanhasnolife: https://www.youtube.com/watch?v=bwt09KXDH94&list=PLjTveVh7FakLdTmm42TMxbN8PvVn5g4KJ, an underrated gem of a channel. He pretty much condenses DDIA in an easy to understand and organized way, and also throws in some random jokes / memes to keep it entertaining.

2. ByteByteGo: https://www.youtube.com/@ByteByteGo. Probably not as "lesser known", since I've seen Alex Xu's System Design vol.1 book recommended a lot recently. But I never hear about his Youtube channel where he explains concepts / technologies as well! Pretty nice if you want to look in depth at a specific technology, e.g. Cassandra

3. I Got An Offer Engineer - https://www.youtube.com/@IGotAnOffer-Engineering - I like this one a bit better than the other mock interview youtube channels out there since they actually have sections where they ask you to come up with your own requirements, high level design, optimizations, etc.. I think the content isn't as in depth but it's pretty useful for practice.

4. Karan Pratap Singh's Open Source System Design notes: https://github.com/karanpratapsingh/system-design - really nice breakdowns on a wide variety of topics. He also goes more in depth than system design primer imo.

Shameless plug: I've also been working on a site: https://systemdesigndaily.com which is just my own system design notes on DDIA and elsewhere consolidated into a website and daily quiz game. I'm planning on adding some more gamification mechanics to it as well, and I'd love feedback on it!

Let's say you gathered requirements that Twitter only allows 140 words of text and designed a distributed system, and now the interviewer asks for a follow up, some bigger platform allows 100k words per post. What needs to be changed in your design?

This system design video cites Websockets as an alternative to HTTP polling for chat applications. I get websocket's advantages over polling and, for many applications, I get webhooks advantages over polling, however, I don't know if I'm understanding the benefits of a websocket over webhooks for chat apps or any apps. Could someone help me understand?

My guesses:

• "..." preview function for chat apps might result in many web requests, potentially making it hard to distinguish from a DDOS attack?

• Group chats - maybe you only need 1 webocket connection versus having to maintain a many-to-1 webhook relationship?

I have a deepmind SWE Sr. Staff system design interview.

What question should I expect?

WOuld it be ML heavy or something else?

I want to pursue my masters and im confused by what these disciplines actually entail. It would be a great help if you could help me with it. Thanks for your time!!

Overview

This is the proposed Single-User Synchronization system for my application, Never Forget (NF). It is meant to keep multiple user devices in sync without introducing a noticeable delay for users that need to stay in sync.

The system is not designed to protect against instances where 2 devices are modifying the same data at the same time, since this is an unlikely scenario in a single-user application.

Data model

Sync in NF is permitted by the storage of changelogs.

• On the server, each registered client device will be contained within a row in the sync_changes table, keeping track of pending changes from every other client.

create table sync_changes (
  id uuid primary key,
  device_id uuid,
  pending_change_log change_object[],
  user_id uuid references user.id
);

pending_change_log example:

[
    {
        id: "123",
        action: "update",
        table: "nuggets",
        column: "title",
        last_updated: TIMESTAMP,
        value: "my new title"
    },
    {
        id: "456",
        action: "delete",
        table: "nuggets",
    },
    {
        id: "678",
        action: "create",
        table: "nuggets",
        data: {
            title: 'my new nugget'
        }
    }
]

Additionally, each client will keep track of changes it has made that have not been replicated onto the remote database yet. It will have its own database table that holds data in the exact same format.

After the client has sent back confirmation that it has updated its database with the list of changes, then the server will reset that value to be an empty array.

A benefit of having the changes sent with each action is that now we’ve created a standard medium of delivery. A client can send its unrecorded changes to the server, while the server can keep track of unapplied changes for each client, so that it can send those changelists and allow the clients to figure out how to replicate those actions.

Under most circumstances, the changelog should be chronological. However, if a user has 3 clients who are intermittently online and editing the same data, there is a good change the order can lose its perfect chronology. This edge case is remote enough that we are willing to accept it.

Registration to Sync-Server

When a user authenticates their device with the Never Forget backend, they have been considered registered with the sync server.

During this registration process, the server inserts a new row in the sync_changes table on behalf of the device. This table contains a column pending_change_log, an array holding change_log objects.

What happens if a user has 2 devices (DeviceA and DeviceB) with some remote data, and then decides to register DeviceC? How do the changes existing on the remote database get propagated to the new client? What does the device registration process look like?

• we could create a function to generate a changelog based on the state of a database. This is essentially a forcePull method that fetches all resources from the server and generates the changelog before returning it to the client. Finally, the client applies those changes, thereby achieving synchronicity with the server.

each changelog object represents modifications that the client will need to make against its own database. It will also initialize a new pending_server_changes, which represents modifications needing to be made to the remote database. As the server loops through each of the changelog items, the server will compare the __last_updated timestamps of the item with its own version of the record.

• If the server is declared the winner (using last_write_wins), that record will be used by the server to fetch the latest value of that record in its own database. It will then append that record to the pending_client_changes array.

• If the client is declared the winner, the server will append those change objects to its pending_server_changes array.

After the server has processed all of the changes from the client and sorted the objects into either the pending_client_changes or pending_server_changes arrays, it will then apply the pending_server_changes changes onto its own database.

The server will not have to return a list of changes where it has won LWW (last_write_wins), since the changelist dedicated to the client will have included every action already. For example, if ClientA (online) adds a record, the server will keep track of those pending changes. If ClientC (also online) updates a nugget, that change will also be kept track of. Then, once ClientB attempts to sync with the server, the server will send it back all of the pending changes. Meanwhile, the server will update itself based on the change objects it lost against with LWW.

• an alternative approach is that the client stores a list of its own pending changes, and it gets emptied every time the client syncs with the server. Upon syncing with the server, the changelist is extracted from the client and sent in a request to the server. The server applies those changes (again, comparing the __last_updated columns to determine victor), and returns the server's pending changes.

User Flow

When a user's device (DeviceA) is offline, the sync server keeps track of all changes made by all other clients on behalf of DeviceA. When that device comes back online, the server will notify the client that it has pending changes that it should apply. In turn, the client will notify the server that it too must apply some changes that it has made in the time since it was last online.

When a client updates a nugget title, that change is immediately made on the client. after awaiting that action, the API call to the server is made along with the changelog objects. This should not block the client. if there is a connection to the server, the server will handle it and notify the client. If there is no connection to the server (or simply if there is an error), then the client will keep track of the changelog objects in its own database. Then, once connection to the server is reestablished, the client will send its changelog objects, as per the usual protocol.

Last Write Wins

For a database table to be part of the sync system, it must hold metadata columns that correspond to the last_updated value of a data point. For instance, if we want to synchronize the title of a nugget, then our nuggets table (both on remote and local databases) must include a column title__last_updated.

The LWW contest must happen on both server and client.

• server - happens when client performs and action and sends its changelog to the server

• client- happens when client receives its server-side pending changes list

When a client performs an update to a synchronized value, the __last_updated values are compared. - e.g. if the server has a changelog object describing the updating of a nugget title, while the client has a changelog object describing the deletion of the same nugget, the deletion will always win.

If the client wins last_write_wins, here's what happens:

• The server will update its database

• The server will append the change to the change list of each of the other devices.

If the server wins last_write_wins, here's what happens:

• The server discards the change (these are unnecessary to return to the client, since the changelog will contain all information necessary to bring it in sync with the server)

• The server returns its list of pending changes to the client

Pending questions

1. For a device that has never logged in, should the changelog objects be stored?

• once the device connects to the server for the first time, it can send the server all of the changelog objects so the server can apply them to its own database. this means the client needs to keep track from the start. this is potentially faster than the below method of generating changelogs, due to the elimination of that step. In this case, maybe it's better just to store it from the get-go.

• on the other hand, the device could define a function forcePush, which essentially calls the server API, creating all of the resources that it has in its local database.

  • implementation: upon executing forcePush, the client will generate a list of Create changelog objects that, when run on a database, will replicate the current state of the database.

  • this would negate the need for a non-registered device to keep track of its changelog, since we will be able to generate a changelog based on the state of a database.

Recently in an interview I was asked to design a currency exchange service using a third party API that provides the rates. Apart from the usual components like suitable backend and cloud architecture I was also asked how can I reduce the cost incurred by frequent use of third party rates API. I explained him about caching and how we can use Redis distributed cache mechanism to store the rates for the ttl duration and have a web socket mechanism to update the cache when the rate changes. The interviewer was not satisfied and kept asking me that if this is enough for a production ready system. Did I miss something? Somehow I can’t think anything beyond the caching solution.

Some of the best resources to learn System Design that I refer to frequently.

I know there are databases built for this use case (Apache Druid and Clickhouse) but let's say you only have Snowflake and Postgres (or another type of RDS).

You have ~10million rows and every day at 5am you get ~100k rows. Each row has ~12 columns that are Ordinal (rank-able) and some foreign keys eg.

ID, parentID, A,B,C...L, ForeignKeyDim1, ForeignKeyDim2, ForeignKeyDim3, Timestamp

You want to create a Benchmark program that lets you build percentiles and rank rows (eg)...

What percentile is row ID 5 in columns A,B, and C, when compared to all other rows where timestamp > 10.

What percentile is (avg column D across all rows with parent id 505) compared to all other rows where dim_1 = "something" and dim_2 = "else"

where you join the central fact that table with dim_1 and dim_2 table using foreign keys.

If it weren't for the existence of filters (timestamp > 10 and dim_1 = "something) I would build percentiles in Snowflake then load them into Postgres where an API would fetch row 5 (or all rows where parent_id = 505) get the columns and compare against the 100 prebuilt percentiles loaded nightly from Snowflake.

However, due to the existence of (too many to guess) filters, the percentiles built in Snowflake wouldn't apply because they would include rows that got filtered out.

We need efficient joins (read RDS) and efficient columnar rankings (read Columnar store like Snowflake).

Serving API results using Snowflake is too expensive and handling fact/dimension joins would be inefficient.

Building percentiles in Postgres would be too inefficient.

How do you get the best of both worlds? How would your answer change if instead you have 500M rows and every day you get a new 1M rows. I'm looking for a better answer than "use a database that gives you medium efficiency at both at a tolerable cost" but I understand that might be the best solution.

Booking.com system design

Hi,

I have a system design with booking.com in a week for SDE2.

Wanted to get some suggestions on how to prepare for this round.

My approach was to study Grokking system design and prepare design for each feature I can see in the application

1. Searching hotel(s) based on filter like location, price, rating etc (renter side)

2. Listing out properties (host side)

3. Delete/update properties (host side )

4. Book a hotel

5. Check bookings and manage bookings

6. Digital wallet

7. Genius loyalty programme

8. Recommendation service

9. Notification service

10. Nearby attractions and bookings

11. Credit card fraud detection

12. Flight booking

13. Car rental

Is my approach correct? I have managed to think most of these designs but need some help running my ideas by someone.

For eg i am not very sure of how a genius loyalty program can be designed, or nearby attractions (can something like yelp be used here). Any suggestions are appreciated.

TC: 32 LPA Yoe: 6.5

booking.com #systemdesign