9

My work proxy server requires authentication with the Microsoft AD domain user credentials. Everybody knows how it works: If you log in on a Windows workstation, your "Internet Explorer" browser based internet access requests are automatically authenticated (and identified) using your domain login credentials.

I found that Firefox can also authenticate against these proxy servers and long assumed that they "do something special". Recently a colleague installed Linux Mint in a VM and to my surprise he was busy getting updates from the internet. When I asked how he got it wo work he shrugged and said "It just worked"

This prompted me to re-look at the proxy settings. I run Kubuntu (with a mix of G* and K* applications, but I only use the GTK applications when I'm convinced that they are much better than anything K*)

I do still have a copy of Windows guest running in a VirtualBox VM, mainly for Printing and for accessing internal/corporate web sites (Which both requires authentication and identification via MS domain credentials) as well as for changing my domain password every so many days.

So it would be very helpful if I could get [some/most/all] of my Linux applications to work via the proxy server. My most urgent needs are for Akregator and Muon to be able to work. Other applications that may benefit are some apps that auto-update (Eg Virtual Box Extentions) or wrap themselves around a browser (Get More Themes/Wall Papers/etc comes to mind, and the occasional use of wget)

SSH/SCP clients manage to work via the firewall without authentication.

What is the right way (tool and/or procedure) to configure this, ideally in a single location because having to maintain my password in multiple locations is a recipe for getting locked out of my account :-/

Oh, and it would be a dream come true if I could have the equivalent to the Firefox "Quick Proxy" proxy disable/enable utility, eg one click to enable or disable the use of proxy, without heaving to log out and back in, depending on what network I'm on. Actually thinking about it, a utility should be trainable to look at your IP address and know when you need to use the proxy! But I digress.

I imagine running a local proxy server which can dynamically add the authentication and forward to an upstream proxy server when needed may be the only real solution.

Johan
  • 4,088
  • 2
  • 23
  • 30
  • What type of proxy is it? Linux based or windows based? – midnightsteel Apr 02 '13 at 16:53
  • The Proxy Server is Windows based. – Johan Apr 03 '13 at 10:40
  • It sounds like the proxy is in a pass by configuration, and your firewall is allowing outbound web traffic from all hosts. This is how your colleague was able to browse without authentication. Is the proxy server that you are using ISA or TMG? – midnightsteel Apr 03 '13 at 13:45
  • The firewall does not allow outbound from all hosts, otherwise I would just bypass the proxy server. What does "pass by configuration" mean? Let me clarify: From my Linux workstation, the only application that can access the internet are a) Firefox (using its own proxy configuration and authentication stored in Firefox), as well as applications running in a Windows VM (Note - the windows VM is a domain member and the user authenticates against the domain when logging in) – Johan Apr 03 '13 at 15:49
  • I don't know whether it is ISA or TMG. If the solution is different depending on which, I would be interested in both solutions. – Johan Apr 03 '13 at 15:50
  • With your current set up I dont think that there is much you can do. It sounds like the windows clients are passing kerberos information to the proxy this is how they are able to surf upon login. That leads me to believe that a Microsoft Proxy product is in use (ISA/TMG). I'm not sure if you can mimic this behavior with Linux clients – midnightsteel Apr 05 '13 at 16:26
  • @midnightsteel If that is true How then is Firefox (both on Linux and on Windows) able to work? – Johan Apr 06 '13 at 08:29
  • Without knowing exactly what is going on on your infrastructure it is hard to say – midnightsteel Apr 08 '13 at 16:04
  • Did the Linux Mint colleague do any Kerberos authentication beforehand? The proxy server might be doing SPNEGO (essentially the MS equivalent of SASL). – Bratchley Apr 08 '13 at 16:36
  • @JoelDavis No. I have a feeling he is using some kind of proxy service on the Host OS to aid Non-MS applications to work via the Proxy server. I am planning on walking through the Linux Mint installation procedure with him and asking him to look at his computer to see if there are any clues there. – Johan Apr 08 '13 at 18:44
  • Your question is too chatty, you should simplify it IMHO. – Totor Apr 09 '13 at 23:27
  • First - it is not true that this process is automatic - even not on windows. Second: Can you authenticate against that proxy manually (i.e. by typing username and windows-password) using firefox on Linux? – Nils Apr 10 '13 at 12:51
  • Yes, Firefox works on Linux, using its own proxy configuration and authentication details. This is why I hope that other software should be able to authenticate too. – Johan Apr 10 '13 at 13:11
  • @Totor I will re-write the question. I'm just pressed for time. – Johan Apr 11 '13 at 11:17
  • I have confirmed that Akregator tries to use the KDE system proxy settings. Basically when having a direct internet conenction and the KDE Global Proxy setting is set to None, Akregator works (can get updates). However when the KDE proxy specifies the local cntlm proxy server, Akregator times out, indicating cntlm is still trying to forward to its upstream proxy server. When actually behind an NTLM proxy server, Akregator fails to get updates regardless of the KDE global proxy setting. – Johan Apr 11 '13 at 14:14
  • @Johan: Did you try ntlmaps? Also, does your proxyserver give you a .pac file? you can use it with ease in firefox and other browsers. – positron Apr 15 '13 at 10:50
  • My proxy server is a computer locked in a computer room, it can't give me files, coffee, or anything else for that matters. On a more serious note: How can I tell? – Johan Apr 15 '13 at 11:10
  • I have mostly solved the problem using cntlm and a mixture of KDE global and application specific proxy settings. I will detail the solution in an answer. – Johan Apr 15 '13 at 11:14
  • @Johan: You can tell cos, my company gives me a .pac file [http://en.wikipedia.org/wiki/Proxy_auto-config] to use. Other than browsers, I simply export http_proxy and https_proxy env variables. That works for all my applications except for apt-get, for which I need to update the same in /etc/apt/apt.conf. – positron Apr 15 '13 at 13:21
  • @positron I have tried http_proxy (and friends), both through user's .bashrc as well as through system initialization scripts /etc/profile and /etc/bash.bashrc but had no success. wget seems to be the only one using it, and updating .wgetrc is in fact more dynamic! In between "real work" demand on my time I'm busy documenting my working solution. cntlm is central to this. – Johan Apr 16 '13 at 06:31
  • @Johan: Well, I just use these env variables. But most of my work is centered around git. For Git this works, may be your requirements are advanced ;) – positron Apr 16 '13 at 06:33

3 Answers3

8

cntlm provides a solution, along with some configuration.

The basic steps to follow are:

  1. Install cntlm.
  2. Edit its configuration file at /etc/cntlm.conf, the comments included makes it easy enough.
  3. Add a proxy server (or two).
  4. Specify your user name and NT domain name in the appropriate places, and remove the password entry.
  5. Start the cntlm service, eg via

    $ sudo /etc/init.d/cntlm start
    
  6. cntlm provides a way to test the proxy and generate a hash from your password - eg by the command (as root)

    $ cntlm -I -M http://www.test.com
    
  7. cntlm will prompt for your domain password. Then it will test different authentication mechanisms against the configured proxy server. When a working method is found, it prints two lines which needs to go into its configuration file.

  8. Stop the cntlm server and add the lines obtained in step 6 above to /etc/cntlm.conf.
  9. Restart cntlm

Now cntlm is running and ready to be used. A number of places can be used to configure various programs to use it. cntlm will then transparently add NT domain authentication tokens to outgoing packets and forward them on to the configured proxy server.

Using Qt/KDE

For Qt / KDE native applications, specify "Use manually configured proxy server" in the KDE System Settings -> Network Settings -> Proxy settings. The proxy is specified as http://localhost with the port 3128 (The default for cntlm unless you changed it). These applications are updated dynamically with new settings and no restart or logout/login is required to update the settings.

Dropbox & Google clients

Many applications can use shell environment variables. Noteworthy here are the Dropbox and Google Earth clients. For these applications use shell environment variables like these:

no_proxy=localhost,127.0.0.0/8,*.local
NO_PROXY=localhost,127.0.0.0/8,*.local
all_proxy=socks://localhost:3128/
ALL_PROXY=socks://localhost:3128
http_proxy=http://localhost:3128
HTTP_PROXY=http://localhost:3128
ftp_proxy=http://localhost:3128
FTP_PROXY=http://localhost:3128
https_proxy=http://localhost:3128
HTTPS_PROXY=http://localhost:3128

s3cmd, curl, & wget

Note: Some applications will use only the lower-case names, others only the upper-case names, some will first try the one, then the other.

s3cmd (The Amazon S3 client), curl and wget can additionally be configured via their own configuration files if desired. This is handy because they read their own config files on every invocation. Since these programs are typically short-lived (A single invocation exists after it completes a download) that is very useful.

The format for s3cmd in ~/.s3cfg is:

proxy_host = localhost
proxy_port = 3128

The format for wget in ~/.wgetrc is:

https_proxy = http://localhost:3128
http_proxy = http://localhost:3128
ftp_proxy = http://localhost:3128

The format for curl in ~/.curlrc is:

proxy = localhost:3128

On the other hand editing shell profile or other environment configuration files typically require a restart, log out-and-back-in, or similar. It is worth investigating /etc/environment, ~/.pam_environment, ~/.kde/env/proxy.sh etc as these are standard places to set proxy settings configured via shell environment variables, particularly to affect all users and services on the system.

I also understand that it is possible to change environment settings on a per-application basis using their respective .desktop files but have not tried it (successfully) yet.

VirtualBox

VirtualBox can be configured to use a proxy (for example to check for and to download software updates) using its GUI or using the command:

$ VBoxManage setextradata global GUI/ProxySettings \
    "proxyEnabled,localhost,3128,authDisabled,,"

For completeness' sake, to disable it use:

$ VBoxManage setextradata global GUI/ProxySettings \    
    "proxyDisabled,,,authDisabled,,"

Firefox

For Firefox I use the QuickProxy addon. Firefox itself if configured manually to use the proxy server on the local machine, so QuickProxy merely enable/disable the setting.

APT

APT (used in the background by synaptic, muon and friends) uses a configuration file in /etc/apt/apt.conf.d/, eg 00proxy Enabling the proxy for APT is done using lines like:

Acquire::http::Proxy "http://localhost:3128";
#Acquire::ftp::proxy "ftp://localhost:3128/";
#Acquire::https::proxy "https://localhost:3128/";

Note: add-apt-repository uses the root profile, or you may configure sudo to allow all the http*_proxy settings to fall through.

A script to rule them all

FWIW I am right now in the process of writing a modular script to enable/disable proxy for numerous programs. I have so far written the following modules:

$ ls -lF proxymanager/modules/
total 60
-rwxr-xr-x 1 root root  919 Oct  8 17:27 apt*
-rwxr-xr-x 1 root root 1037 Oct  8 13:10 bashrc*
-rwxr-xr-x 1 root root  391 Oct  8 12:18 cntlm*
-rwxr-xr-x 1 root root  684 Oct  8 12:58 curl*
-rwxr-xr-x 1 root root  609 Oct  8 13:02 dropbox*
-rwxr-xr-x 1 root root  672 Oct  8 12:18 gnome*
-rwxr-xr-x 1 root root  691 Oct  8 12:18 kde*
-rwxr-xr-x 1 root root  689 Oct  8 13:03 root_bashrc*
-rwxr-xr-x 1 root root  691 Oct  8 13:03 root_curl*
-rwxr-xr-x 1 root root  827 Oct  8 13:03 s3cmd*
-rwxr-xr-x 1 root root  454 Oct  8 13:03 survive_reboot*
-rwxr-xr-x 1 root root  860 Oct  8 13:06 suse-sysproxy*
-rwxr-xr-x 1 root root  653 Oct  8 12:46 sysenvironment*
-rwxr-xr-x 1 root root  465 Oct  8 13:04 virtualbox*
-rwxr-xr-x 1 root root  573 Oct  8 13:04 wgetrc*

As well as a control application. These will hopefully soon be moved into a github or other online home.

slm
  • 363,520
  • 117
  • 767
  • 871
Johan
  • 4,088
  • 2
  • 23
  • 30
  • This has since moved to GitHub, gained more modules, and is slowly but surely approaching usable form. Daring and adventurous types may find this useful: https://github.com/Tahaan/proxymanager – Johan Apr 08 '14 at 10:27
  • New thing to learn.Thanks.How can I pass the password to cntlm in command line... I don't want any prompt asking to enter the password? – Thushi Feb 04 '15 at 04:08
2

From my Linux workstation, the only application that can access the internet are a) Firefox (using its own proxy configuration and authentication stored in Firefox), as well as applications running in a Windows VM (Note - the windows VM is a domain member and the user authenticates against the domain when logging in)

Solution option: Run a web proxy on your windows VM. Setup your system to use that instance as your proxy.

Since your Windows VM is already authenticated and traffic is allowed through it, setting up a SOCKS proxy on that VM instance will centralized your authentication needs. If it is just for you and your boxes, this should be fine and is probably fairly straightforward.

Piggybacking on this idea is to get an SSHD daemon running on the Windows VM so you can do things like SSH SOCKS tunnels from your other boxes through the VM:

ssh -D 1080 windows-user@windows-vm

For those apps that may have problems or where you don't want to reconfigure the apps, you can make use of sshtunnel, which will setup iptables rules to route traffic. Works for Linux and Mac systems.

If you need to avoid installing a proxy on the Windows VM itself, you can setup a Squid proxy box configured to authenticate itself against the windows AD. A guide on doing that located here:

Solution option: Squid Proxy Authenticated via AD/NTLM

http://techmiso.com/1934/howto-install-squid-web-proxy-server-with-active-directory-authentication/ (dead link)

Another NTLM proxy solution, though I think this one actually runs on a Windows machine:

Solution option: NTLM proxy http://cntlm.sourceforge.net/

nihiser
  • 103
  • 4
Wing Tang Wong
  • 326
  • 1
  • 5
  • I am investigating cntlm already, it happens to be in the standard ubuntu repositories too. – Johan Apr 11 '13 at 06:27
  • 1
    The cntlm solution appears to be promising. So far it allows wget and chromium to work. Wget works by having the cntlm address:port in the .wgetrc file. Chromium works being configured to use what it calls the "computer's system proxy settings", where I have configured the proxy in the KDE system settings->Proxy->Manual Configuration. Chromium uses this dynamically, ie if I change the setting in KDE, Chromium gets affected without having to restart chromium. TBC – Johan Apr 11 '13 at 11:06
  • So far this is still NOT working for Konqueror, Muon or Akregator, none of which exposes Proxy settings directly. I believe Muon uses apt-get in the background, which is configurable in a way similar to wget. I do not understand the difference between KDE "Manually specified Proxy" and KDE "System Proxy" configuration options, this may be key to getting it to work for all applications. I also notice that the KDE settings are not being populated into shell scripts, even after log-out-and-back-in, which is disappointing. – Johan Apr 11 '13 at 11:09
  • The proxy-through-MS-Windows solution is not acceptable to me. Others may feel different but running a whole extra OS instance just to add authentication to outbound network packets is a bit too much of a resource hog and doesn't remove my dependency on MS Windows. – Johan Apr 11 '13 at 11:20
  • @Johan I only suggested the proxy-through-MS-Windows as a possible solution. I've been in situations where I had to do with what I had. :( I did a quick search on Konqueror proxy settings and it seems that depending on the version/environment, the settings vary wildly, in some cases, needing to set the proxy settings as root for them to stick globally. :( For the shell environments, I'll generally set the global proxies in the /etc/profile* config files, then make sure the account rc/profile files reference the global one. – Wing Tang Wong Apr 11 '13 at 17:28
  • maybe you can edit your answer to focus on the cntlm proposal, explain what config files to set in various applications, etc, and add pointers to solutions in case that is not possible. I'd like the focus of the answer to be around the Linux based solution. – Johan Apr 12 '13 at 07:09
  • Unfortunately, I don't have a Windows machine handy. Downloading an eval of the Win2k12 server and will test it out. Interestingly enough, cntlm is available on Mac OS X via Homebrew(awesome!) – Wing Tang Wong Apr 13 '13 at 04:42
  • @Johan quick update. No love on the Win2k12 install on my local box. Needs 25GB to install, but I don't have that much resource free, given the other projects I'm working on. :( So, got no Windows auth server to test against and cannot recommend any further beyond the cntlm example config file that comes with the program. – Wing Tang Wong Apr 15 '13 at 23:36
  • 1
    I've solved the problem using cntlm as per the suggestion above, and highly recommend it. The issue I had with most KDE applications not using the KDE global setting was due to incorrect format used in specifying the proxy server in the KDE proxy settings. Incidentally Chromium appears to not be sensitive to this (I left out the `http://` when configuring the proxy) – Johan Oct 09 '13 at 11:26
0

Proxy authentication with ntlm is explained well in the below links.

But the problem with ntlm is you have to update the cntlm configuration file many times a day. Consider a situation where different users share a common machine and they have to update their AD password in the cntlm config file each time they login to the system. Perhaps there's a method available to automatically update the cntlm configuration file whenever a user is logged in.

slm
  • 363,520
  • 117
  • 767
  • 871
  • I only update my cntlm config file when my NTLM proxy password (The domain password) changes. – Johan May 07 '15 at 07:33