Questions tagged [distributed-systems]
The distributed-systems tag has no summary.
15 questions
0 votes
0 answers
35 views
Limiting Access to logs on a specific cluster to Grafana Admins
I have 3 cluster (A-B-C), I installed loki on cluster A using the grafana/loki helm chart, and on cluster B and C I have promtail installed pushing logs to cluster A, how can I make logs collected ...
- 1
0 votes
0 answers
62 views
Kubernetes networking somehow breaks ROS2 message passing
I have 2 identical docker containers running on separate computers in which I want to run ROS2 nodes communicating with each other (via a VPN). When I run them as docker containers there is no problem ...
0 votes
2 answers
175 views
How to implement simple distributed configuration system
I have multiple servers running in data centers around the world. Each server has its own configuraiton database, and cli command needs to be executed to update in-memory cache. I want to automate ...
- 111
1 vote
1 answer
404 views
Distrubuted storage PVC on k3s using OpenEBS is stuck in a pending status while provisioning - waiting on external provisioning, not sure why
I'm trying to setup distributed raid 1+0 storage on my k3s cluster running of 5 raspberry pi 4s running armbian (jammy). Currently I'm seeing the event waiting for a volume to be created, either by ...
- 21
0 votes
1 answer
1k views
s3: reading parquet files with a subset of columns as argument; does this reduce data transfer?
I have set of large tables stored as parquets on s3. In python, I'm using: pd.read_parquet(...,columns=columns) I'm reading the files directly from s3, without any database engine whatsoever for ...
1 vote
1 answer
42 views
Proper technologies to create a horizontably scalable index search
I am sorry for the newbye question. Im trying to figure out the best architecture / set of technologies to implement some very specific system: I have several databases indexes (used for searching ...
1 vote
2 answers
193 views
Dynamic environments per client, which is the best approach, if any?
I don't know how to explain this problem better, but I'll try to explain it clearly: Where I work our customers are companies, with lots of users We offer a SaaS solution for them to manage stuff For ...
4 votes
1 answer
431 views
Necessity of mirroring git repos for small team considering switching to Azure
Question: Are there any benefits to mirroring our git repos outside of Azure devops for a small (~7) team of developers who all work in the same office? Background Information I'm a developer who ...
- 141
10 votes
5 answers
6k views
What is self hosted version control system?
What is self hosted version control system and how is it different from github? I mean if I use self-hosted VCS then where does my code go? I mean: does it get uploaded to the server, or it is saved ...
- 101
-2 votes
2 answers
340 views
How do you monitor status and view logs for multiple distributed java applications at a centralized place?
We have multiple java applications running on distributed hosts in our system. As theses applications are processing huge amount of data, it is designed this way. However, it had been a challenge in ...
- 129
2 votes
2 answers
138 views
Identifying a benchmark to compare overall performance of VM virtualization vs hardware
Which performance index is most suitable to find out real performance of your VM limited by sharing and quotas? For example, if you have a physical server with 4 cores and 16 GB RAM, its performance ...
- 6,792
1 vote
2 answers
284 views
Automated Docker swarm reinitialization and token distribution
Imagine you want to reset your Docker swarm from time to time. How do you distribute join tokens to those zillions of nodes? How much automation is possible here?
- 6,792
4 votes
1 answer
257 views
How to route to specific, of multiple datacenters, with a single domain
The data of each customer only exist in one of several global datacenters, that all answer to the same hostname (ex. api.company.com) Only a single domain for all customers globally I want to make ...
- 171
7 votes
3 answers
3k views
How to ensure equitable distribution of SQS messages in a distributed systems setting?
I have multiple servers, each having a script polling an SQS queue [all polling the same queue]. So, is there any way I can ensure an equitable distribution of messages to all those clients [i.e. my ...
- 2,816
16 votes
3 answers
5k views
How to avoid "retry storms" in distributed services?
A "retry storm" is caused when clients are configured to retry a set number of times before giving up, a retry policy is necessary because of packets loss will occur in normal operation of a service. ...
- 11.8k