Man-in-the-Middle (MITM) Firewall

Be sure to check out Troubleshooting SSL Issues as it provides a quick step by step guide that might fix your problem without getting too technical. If this is not enough to solve your issues, this page offers more in depth information.

Introduction

In this article, we are going to talk about a specific type of company firewall setup that is gaining a lot of traction. The main purpose is to give you enough information to start the discussions with your IT/firewall provider.

It will cover the most common symptoms, their reasons, and possible solutions.

👉 If you have any hesitations, we recommend getting your IT and Robocorp support in a meeting as soon as possible.
With the current options, the solutions are usually relatively quick, but finding the correct terminology to avoid confusion is critical.

Before starting, it is good to know that firewall providers use multiple names for the same functionality. Below are a few to help find the correct term with your IT:
Transparent HTTPS, HTTPS Inspection, SSL/TLS Inspection, SSL Interception, SSL Forward Proxy, SSL Decryption

Problem / Symptoms

Seeing SSL verification failed. when setting up tooling or trying to run a robot.

Almost all tools expect HTTP traffic to be encrypted (expect end-to-end HTTPS for security). Now, with an MITM firewall, the firewall needs to be able to scan the content of the traffic, but to do that, it needs to break the regular HTTPS traffic to be able to do so.

For the tools, this is visible by the fact that the HTTPS traffic is encrypted and signed by a self-signed certificate. These types of certificates cannot be verified against any global public root Certificate Authorities (CAs), so by default, the tools will either warn or refuse to work with this kind of traffic.

Reason and background

The reasons why these types of firewalls are becoming more and more popular:

• Enterprises need to provide more access to users and outgoing traffic needs to be scanned to avoid leaking PII data or company IP information.
• Malware is becoming increasingly active and relevant, requiring more than static rules and passive scanning.
• The firewalls are also starting to leverage AI, dynamic rules, and restrictions, enabling faster reactions to potential security breaches.

🤔 So why is this a problem and why is my browser working just fine?

Simply put, applications and tools built by or just for Microsoft Windows can directly use the OS certificate store. The same applies for macOS applications. It is standard practice for the firewall providers to put the self-signed CA certificates into the system certificate store, and then the tools can verify against that. For open-source frameworks and tools like Python, NodeJS, etc., integrating into these platform-specific certificate sources is typically quite a significant effort to make. By default, they do not have these supports in place or activated.

Solution

The solutions differ quite drastically regarding the security implications, but it is critical to understand the Situation.

1. The firewall certificate(s) in the OS certificate store

It should be standard practice to import the extra certificates into the certificate store automatically. Still, it is good to verify from the IT/firewall provider.

The system OS store is a bit different based on the OS your are running:

• On Windows certmgr.msc -tool normally opens the Certificate manager
• On macOS this is called Keychain or Keychain Access app
• On Linux certificate management is a bit more manual and the location depends on your distro:
  /etc/ssl/certs/ca-certificates.crt // Debian/Ubuntu/Gentoo etc. /etc/pki/tls/certs/ca-bundle.crt // Fedora/RHEL 6 /etc/ssl/ca-bundle.pem // OpenSUSE /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem // CentOS/RHEL 7 /etc/ssl/cert.pem // Alpine Linux

Again the firewall/proxy provider should have guidance on these for your IT

While SSL errors are different from firewall inbound / outbound rules and whitelisting, it is good to note that most MITM firewall setups have very strict rules that block any traffic by default. If the firewall completely blocks the traffic, you will not see the SSL errors as the traffic just gets blocked. Normally, you will need some outbound whitelist rules.
Robocorp tools only require outbound traffic; please check out our firewall rules page.

2. Get the tools and automations to use the system certificate store

As of 2023-09-18, Robocorp libraries and templates provide versions that enable Python implementations to use the system certificate store without extra changes to the automation logic.

As support is quite a new thing in Python tools, you do need to update your automation dependencies (conda.yaml) as follows:

Python 3.10.12 is enables the use of robocorp-truststore, which in turn enables:
--use-feature=truststore -flag that guides pip also to leverage system certificates.

3. Troubleshoot and corner cases

As Robocorp, we cannot cover every possible tool out there. Also, if IT setup changes take time, you may need some temporary solutions. This section has many tools and processes, but most should be able to work without any of these.

Collect extra CA Bundle files

Most open-source tools support creating specific certificate bundle files containing the "extra certificates". The problem with these is the collection and management of these files. The new wave of firewalls is looking to make the self-signed certificates shorter-lived and more specific, which will mean more maintenance for the file approach.

RCC in the Robocorp toolchain supports this via profiles, but you need to contact your IT to get the certificates in the correct format.

Bypass rules

If you hit a toolchain that does not provide a way to mitigate SSL errors, you can ask for a bypass rule for a specific website. In this case, the traffic stays encrypted from end to end with normal certificates, so the MITM part gets bypassed.

This is a temporary solution, and we recommend notifying Robocorp Support of the tool and website that is hitting this case so we could also check if there is anything we can do to help.

Disabling SSL verifications

⚠️ SSL disabling should ONLY be done temporarily and with the permission of your IT.

In this solution, you are stating (usually to a specific tool that it should ignore any SSL errors and warnings). Most tools support some environment variables to turn off SSL warnings. RCC in the Robocorp toolchain provides a way to disable SSL checks via profiles, but we can only affect the tools we know about.

Again, do not hesitate to contact Robocorp Support if you end up at this level.