Cloudera Docs

Proxy setting best practices

Learn about best practices for using environment variables to propagate proxy settings.

In Kubernetes, proxy settings can be propagated to pods through the use of environment variables in pod spec configurations or through ConfigMaps or Secrets that are mounted as volumes within pods. This allows the proxy settings to be passed down to individual containers within the pods. However, it's important to note that not all applications may automatically inherit these settings, and some may require additional configuration within the container image or application code to properly utilize the proxy settings.

It's important to note that the use of no_proxy or NO_PROXY environment variables to bypass proxy settings may not be consistently respected by all third-party libraries or applications. While some libraries or applications may automatically honor these settings, others may not.

Many popular libraries and frameworks in various programming languages, such as Python, Java and Node.js, have their own way of handling proxy settings, which may not necessarily rely on the no_proxy or NO_PROXY environment variables. These libraries may have their own configuration files or internal settings that dictate how they handle proxy configurations, and these settings may not always align with the no_proxy or NO_PROXY environment variables set at the system or project level.

As a result, it's important to be aware that relying solely on no_proxy or NO_PROXY environment variables may not provide consistent results across all libraries or applications used in a project. In some cases, it may be necessary to clear out the http_proxy, https_proxy, no_proxy, or NO_PROXY environment variables in your project's environment variables or configuration files to ensure that the third-party libraries or applications do not attempt to apply proxy settings at all.

To ensure that proxy settings are consistently respected across all libraries and applications used in a project, it's recommended to carefully review the documentation and configuration options of each library or application, and configure them accordingly.

However, if you ever have to manage a stack written in multiple languages, you might need to consider some best practices for setting HTTP proxy configurations:

For http_proxy and https_proxy:
  • Use lowercase form. HTTP_PROXY is not always supported or recommended.
  • If you absolutely must use the uppercase form as well, be sure both versions share the same value.
For no_proxy:
  • Use lowercase form.
  • Use comma-separated hostname:port values.
  • IP addresses are acceptable, but hostnames are never resolved.
  • Suffixes are always matched (for example, example.com will match test.example.com).
  • If top-level domains need to be matched, avoid using a leading dot (.).
  • Avoid using CIDR matching since only the Go and Ruby languages support that.

PIP installs with Proxy

When using pip to install packages from external sources via a proxy, it is generally recommended to perform this installation in a separate session (project session), utilizing the existing proxy rules. Proxy settings, including proxy server addresses, usernames, passwords, and other configurations, are typically environment-specific. By using a separate session, you can ensure that the appropriate proxy settings are used for the specific installation task. This helps to avoid conflicts or misconfigurations with your main session's proxy settings, which may be required for other tasks or applications.

To resolve Python communication issues between pods with proxy setup, in Project Settings > Advanced add the four environmental vatiables (HTTP_PROXY, http_proxy, HTTPS_PROXY, https_proxy) set to empty values. This will allow Python to run and use kubedns and kubeproxy properly.

Environment variables in the customer environment:
  • HTTP_PROXY
  • http_proxy
  • HTTPS_PROXY
  • https_proxy
  • NO_PROXY
  • no_proxy
  • ALL_PROXY

The workaround of adding these environment variables in the project session helps resolve the proxy issue. Since Python does not support CIDR blocks for no_proxy, the request automatically gets directed to http_proxy or htpps_proxy, which causes the failure and prevents jobs from completing.