[Mac/Python] 웹 크롤링 준비 | pip로 beautifulsoup4, requests 패키지 설치

시작에 앞서, 결론만 보고싶으면 가장 마지막

구분선 아래를 보면 된다

나는 PyCharm을 사용중이고,

pip로 beautifulsoup4 패키지를 설치하려고 하는데

계속 에러가 나는 상태다

구글링해서 pip를 설치하고 패키지 설치를 했는데,

# pip 설치
brew install pip

# beautifulsoup4 패키지 설치
pip install beautifulsoup4

pip가 없다고 뜬다?

Python 3.12 버전이라 그런가 해서

pip3로 다시 해봤다
(Python2 전과 후 버전 구분을 위해 pip2, pip3 이렇게 쓴다고 한다)

우선 pip3 입력해서 아래와 같이 뜨면 제대로 설치되어있는거다.

Usage:   
  pip3 <command> [options]

Commands:
  install                     Install packages.
  download                    Download packages.
  uninstall                   Uninstall packages.
  freeze                      Output installed packages in requirements format.
  inspect                     Inspect the python environment.
  list                        List installed packages.
  show                        Show information about installed packages.
  check                       Verify installed packages have compatible dependencies.
  config                      Manage local and global configuration.
  search                      Search PyPI for packages.
  cache                       Inspect and manage pip's wheel cache.
  index                       Inspect information available from package indexes.
  wheel                       Build wheels from your requirements.
  hash                        Compute hashes of package archives.
  completion                  A helper command used for command completion.
  debug                       Show information useful for debugging.
  help                        Show help for commands.

General Options:
  -h, --help                  Show help.
  --debug                     Let unhandled exceptions propagate outside the main subroutine,
                              instead of logging them to stderr.
  --isolated                  Run pip in an isolated mode, ignoring environment variables and
                              user configuration.
  --require-virtualenv        Allow pip to only run in a virtual environment; exit with an error
                              otherwise.
  --python <python>           Run pip with the specified Python interpreter.
  -v, --verbose               Give more output. Option is additive, and can be used up to 3
                              times.
  -V, --version               Show version and exit.
  -q, --quiet                 Give less output. Option is additive, and can be used up to 3 times
                              (corresponding to WARNING, ERROR, and CRITICAL logging levels).
  --log <path>                Path to a verbose appending log.
  --no-input                  Disable prompting for input.
  --keyring-provider <keyring_provider>
                              Enable the credential lookup via the keyring library if user input
                              is allowed. Specify which mechanism to use [disabled, import,
                              subprocess]. (default: disabled)
  --proxy <proxy>             Specify a proxy in the form
                              scheme://[user:passwd@]proxy.server:port.
  --retries <retries>         Maximum number of retries each connection should attempt (default 5
                              times).
  --timeout <sec>             Set the socket timeout (default 15 seconds).
  --exists-action <action>    Default action when a path already exists: (s)witch, (i)gnore,
                              (w)ipe, (b)ackup, (a)bort.
  --trusted-host <hostname>   Mark this host or host:port pair as trusted, even though it does
                              not have valid or any HTTPS.
  --cert <path>               Path to PEM-encoded CA certificate bundle. If provided, overrides
                              the default. See 'SSL Certificate Verification' in pip
                              documentation for more information.
  --client-cert <path>        Path to SSL client certificate, a single file containing the
                              private key and the certificate in PEM format.
  --cache-dir <dir>           Store the cache data in <dir>.
  --no-cache-dir              Disable the cache.
  --disable-pip-version-check
                              Don't periodically check PyPI to determine whether a new version of
                              pip is available for download. Implied with --no-index.
  --no-color                  Suppress colored output.
  --no-python-version-warning
                              Silence deprecation warnings for upcoming unsupported Pythons.
  --use-feature <feature>     Enable new functionality, that may be backward incompatible.
  --use-deprecated <feature>  Enable deprecated functionality, that will be removed in the
                              future.

또 에러가 났다.

# 패키지 다운로드
pip3 install beautifulsoup4

# 결과
× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
    xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:
    
    python3 -m venv path/to/venv
    source path/to/venv/bin/activate
    python3 -m pip install xyz
    
    If you wish to install a Python application that isn't in Homebrew,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. You can install pipx with
    
    brew install pipx
    
    You may restore the old behavior of pip by passing
    the '--break-system-packages' flag to pip, or by adding
    'break-system-packages = true' to your pip.conf file. The latter
    will permanently disable this error.
    
    If you disable this error, we STRONGLY recommend that you additionally
    pass the '--user' flag to pip, or set 'user = true' in your pip.conf
    file. Failure to do this can result in a broken Homebrew installation.
    
    Read more about this behavior here: <https://peps.python.org/pep-0668/>

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

[notice] A new release of pip is available: 24.0 -> 24.2
[notice] To update, run: python3.12 -m pip install --upgrade pip

현재의 Python 환경이 “externally managed” 상태로,

이로 인해 시스템 전체에 Python 패키지를 직접 설치하는 것이 제한된다는데

macOS와 같은 운영 체제에서 Python 패키지 관리의 일관성을 유지하기 위해

이런 제한을 건다고 한다.

그럼 뭐 어떻게 설치하라고

LLM을 사용해보니 pipx를 사용하면 된다고 해서

# pipx 초기화
pipx ensurepath

# 패키지 다운로드
pipx install beautifulsoup4

했는데 또 에러

pipx는 주로 독립 실행형 Python 애플리케이션을 설치하고 실행하기 위한 도구로,

일반적인 Python 라이브러리(예: beautifulsoup4)를 설치하는 데 사용되지 않는단다

가상환경(venv)을 사용하면 설치할 수 있다고 해서

# 가상환경(venv) 생성
python3 -m venv venv

# venv 활성화
source venv/bin/activate

# venv활성화 상태에서 beautifulsoup4 설치
pip install beautifulsoup4

# requests 설치
pip install requests

이렇게 했더니 설치가 됐다.

설치된 패키지 확인을 하려면