To make third-party or locally-built code available to notebooks and jobs running on your clusters, you can install a library. Libraries can be written in Python, Java, Scala, and R. You can upload Java, Scala, and Python libraries and point to external packages in PyPI, Maven, and CRAN repositories.
Steps to install third party libraries:
Step1: Create Databricks Cluster.
Step2: Select the cluster created.
Step3: Select Libraries => Install New => Select Library Source = "Maven" => Coordinates => Search Packages => Select Maven Central => Search for the package required. Example: (lift json) => Select the version ( net.liftweb » lift-json_2.13 » 3.4.1) required => Install
Successfully installed ( net.liftweb » lift-json_2.13 » 3.4.1) library.
For more details, refer "Azure Databricks - libraries".
Hope this helps. Do let us know if you any further queries.
Thanks for the answer, but that is exactly what I did, however in my case I got the errors mentioned =[
Well, I was very unhappy with this because you used the same type of cluster than me and it worked. So I decided to create a new cluster and try again, then it finally worked. Who knows what happened . . .
I have used same cluster as yours Databricks cluster (Apache Spark 2.4.4, Scala 2.11).