I am trying to run a python-script on an aws ec2-instance using jenkins.
I get the following error:
[ProdTest] $ /bin/sh -xe /tmp/jenkins14047325752732522807.sh
+ python3 prod.py
Traceback (most recent call last):
File "prod.py", line 2, in <module>
import base
File "/home/jenkins-slave-strat/workspace/ProdTest/base.py", line 3, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
Build step 'Execute shell' marked build as failure
Finished: FAILURE
on the ec2 instance:
$ python3 --version
Python 3.7.9
$ pip3 install boto3 --user
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Requirement already satisfied: boto3 in /usr/local/lib/python3.7/site-packages (1.16.25)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.7/site-packages (from boto3) (0.10.0)
Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /usr/local/lib/python3.7/site-packages (from boto3) (0.3.3)
Requirement already satisfied: botocore<1.20.0,>=1.19.25 in /usr/local/lib/python3.7/site-packages (from boto3) (1.19.25)
Requirement already satisfied: urllib3<1.27,>=1.25.4; python_version != "3.4" in /usr/local/lib/python3.7/site-packages (from botocore<1.20.0,>=1.19.25->boto3) (1.26.2)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.7/site-packages (from botocore<1.20.0,>=1.19.25->boto3) (2.8.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.20.0,>=1.19.25->boto3) (1.15.0)
How do i get python to access the packages in /usr/local/lib/python3.7/site-packages ?
Thanks to everyone taking the time!
weirdly, the site-package is included in the PATH. I even got some seemingly contradicting statements when installing packages (in this example numpy): WARNING: The scripts f2py, f2py3 and f2py3.7 are installed in '/usr/local/bin' which is not on PATH. However, when checking the path it gives: $ echo $PATH /usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin Anyways, your solution works and looks like better practice than anything i ve been doing at this point :D thanks!
Yeah. If you're not a data science person you may want to look into Pipenv, but basically any Python developer, even on a single purpose VM, should use virtual environments. It's just a quirk of how the language has fragmented.