5

Looking at tools like tc, wondershaper, htb and comcast, all these tools seem to operate on the level of a network interface or at least a "connection group" for limiting bandwidth. I'd like to not throttle bandwidth for a group of connections, but instead throttle the max rate of individual connections.

Specifically: Is there a tool available that I can use to shape the max download rate of individual HTTP requests?

Details

What I'm looking to do is emulate slow requests to fetch from buckets on S3. I'm seeing that for requests that are away from a data center, download of an individual item is usually slow (<500 kb/s) but downloading in parallel yields download speeds >5 mb/s.

I can probably get part of the way there by adding latency in these requests (which slows down throughout of serial requests but not overall bandwidth), but a more direct solution would be great.

turtlemonvh
  • 196
  • 1
  • 7
  • Hmm, would solving it with a proxy like `squid` or `varnish` be suitable? Those have throttling features or add-ons available. – thrig Dec 24 '16 at 02:07
  • 1
    If I was not mistaken.The tool called Tickle.You can set download/upload bandwidth for individual http request – supriady Dec 24 '16 at 05:08
  • 1
    Thanks @supriady - I'm assuming you meant [trickle](https://wiki.archlinux.org/index.php/Trickle) which looks cool but [doesn't work for golang static binaries](http://stackoverflow.com/questions/40386936/why-doesnt-the-trickle-utility-affect-my-dynamically-linked-golang-program/40387514#40387514), which is unfortunately exactly what I'm testing! – turtlemonvh Dec 26 '16 at 21:49
  • @thrig Looking at squid's capability [here](http://wiki.squid-cache.org/Features/ClientBandwidthLimit) it looks to still operate on the level of IP address (like varnish as suggested by chaminda below). I could possibly inject my own "X-Forwarded-For" headers into each request to trick these tools into *thinking* the requests are from different IP addresses, so I *could* make it work... – turtlemonvh Dec 26 '16 at 22:10

2 Answers2

2

According to your requirement I would like to suggest Varnish and it is a highly regarded HTTP caching server. It sits in front of your web server tier and caches content in RAM so subsequent requests are served as quickly as possible.

Following reference article have been demonstrated nicely that how to install and configure varnish on a web server on CentOs.

There are two configuration files to edit, /etc/sysconfig/varnish and /etc/varnish/default.vcl.

Editing default.vcl you can optimize your bandwith issues. I have already installed on my servers.

As an example configurations are look like follows.

backend web1 {
    .host = "PUBLIC_IP_ADDRESS";
    .port = "80";
    .probe = {
        .url = "/";
        .interval = 5s;
        .timeout = 1s;
        .window = 5;
        .threshold = 3;
    }
}

Further Refernce: Setup Varnish 4 on CentOS 6 as a Caching Server and Load Balancer

Chaminda Bandara
  • 423
  • 1
  • 6
  • 22
  • I'm not seeing how to do request level throttling with varnish. I see there is a module called [libvmod-throttle](https://github.com/nand2/libvmod-throttle) that I can use to throttle per IP address or per HTTP header but even there bandwidth isn't throttled, requests are just rejected after a certain limit. – turtlemonvh Dec 26 '16 at 21:37
  • If you're saying that varnish could improve the root performance issues, unfortunately that isn't true. We have a very low cache hit rate - most of the requests we get are part of a long scan over a wide range of keys. Adding a caching layer like varnish wouldn't help here. – turtlemonvh Dec 26 '16 at 21:38
  • Thanks for your answer @chaminda! It sent me looking in the right direction where I found nginx. – turtlemonvh Dec 26 '16 at 22:32
  • Yes, I checked nginx and it's best than Varnish I think for your requirement. So did you implement that ? @turtlemonvh – Chaminda Bandara Dec 28 '16 at 07:13
1

The limit_rate setting of nginx seems to overcome some of the issues in squid and varnish as recommended by other responders. From the docs:

Limits the rate of response transmission to a client. The rate is specified in bytes per second. The zero value disables rate limiting. The limit is set per a request, and so if a client simultaneously opens two connections, the overall rate will be twice as much as the specified limit.

For my scenario, where I'm looking to limit the rate of download bytes transferred for large files for individual requests without limiting the overall bandwidth for a client, this is exactly what I need.

Squid

Squid's delay pools group clients (usually by IP) and use a bucketed rate limiting. However even the docs say:

You can not limit a single HTTP request's connection speed.

Varnish

Varnish's vmod_vsthrottle (and similarly libvmod-throttle) works off a token bucket algorithm and takes arbitrary keys. The implementation seems very cool, but it looks like there is not a good way to slow down traffic. Instead requests above a limit (in req/s) are responded to with something like a 429 .

turtlemonvh
  • 196
  • 1
  • 7
  • Also in the realm of proxy servers, it looks like [fiddler's `*-trickle-delay`](http://docs.telerik.com/fiddler/KnowledgeBase/FiddlerScript/PerfTesting#performance-testing) flags also allows delaying chunks of data on the KB level for every request matching complex rules. This would also satisfy the requirements of the question. – turtlemonvh Dec 26 '16 at 23:01
  • For those looking to restrict *upload* rate, it looks like [the nginx limit_upload_rate module](https://github.com/cfsego/limit_upload_rate) (not included in nginx core) would help. – turtlemonvh Dec 26 '16 at 23:02