Docker – tsmx https://tsmx.net pragmatic IT Wed, 19 Jun 2024 05:30:40 +0000 en-US hourly 1 https://wordpress.org/?v=6.6.2 https://tsmx.net/wp-content/uploads/2020/09/cropped-tsmx-klein_transparent-2-32x32.png Docker – tsmx https://tsmx.net 32 32 Using SQL Developer with PostgreSQL https://tsmx.net/using-sqldeveloper-with-postgresql/ Tue, 18 Jun 2024 20:36:51 +0000 https://tsmx.net/?p=2928 Read more]]> Quick guide on how to connect to a PostgreSQL database using Oracle SQL Developer.

Also it’s primary usage is for Oracle DB, SQL Developer also a good tool for managing other databases like PostgreSQL and you can benefit from known UX/UI. I assume you already have a local installation of SQL Developer and a PostgreSQL up & running at localhost. If you don’t have a local PostgreSQL DB, have look on this article how to achieve this in minutes using Docker.

Preparing SQL Developer to connect to PostgreSQL

To enable SQL Developer to connect to a Postgres DB, get the offical PostgreSQL JDBC driver first. Save the downloaded jar file in an appropriate folder like /opt/postgres and set the permissions so that SQLDeveloper is able to read the file.

In SQL Developer navigate to Tools --> Preferences and there to Database --> Third Party JDBC Drivers. Click on Add Entry and search for the Postgres JDBC jar.

sqldeveloper-postgres-driver

After restarting SQL Developer you have PostgreSQL available in the database type dropdown for a new connection.

sqldeveloper-postgres-new-connection

Create a connection to the PostgreSQL database

Having that, create a new PostgreSQL connection to localhost port 5432 with user postgres and the default password postgres. If you have started PostgreSQL using Docker, provide the password set in the POSTGRES_PASSWORD variable of the Docker run command.

sqldeveloper-postgres-new-connection-2

After saving and connect you are ready to use your PostgreSQL DB in SQL Developer.

sqldeveloper-postgres-connected

Connecting when username does not equal the database name

By default, the connection in SQL Developer is made to a PostgreSQL database named exactly like the user.

Supposing you have a user testuser (without having an own database named the same) and want to connect to a database called testdb. After entering the credentials for that user neither the Choose Database dropdown was populated nor did the button itself worked for me.

The trick – found in this article on StackOverflow – when connecting to a database with a different name is to add the databse name after a slash to the hostname and finally add a question mark at the end, like so: localhost/testdb?.

sqldeveloper-postgres-different-dbname

That’s it. Have fun using SQL Developer with your PostgreSQL DB šŸ™‚

Useful links

]]>
COPY data between localhost and PostgreSQL running with Docker https://tsmx.net/copy-data-postgresql-docker/ Tue, 04 Jun 2024 20:17:16 +0000 https://tsmx.net/?p=2936 Read more]]> A short article showing how to easily use the COPY command to move data between localhost and a PostgreSQL database running with Docker.

With the COPY command PostgreSQL povides an easy way to move data between the database and the local file system. In this article we show how to use this command when PostgreSQL is running with Docker. This is especially useful if you want to move large amounts of data, e.g. populating a database with mass-data.

Mounting a data transfer volume to the PostgreSQL container

In this article I assume you are familiar on how to run PostgreSQL with Docker using a separate user to manage permissions for local mounted volumes. In our example the user is named postgres and has ID 1002.

To be able to transfer data between PostgreSQL in Docker and the host file system, we’ll need a local directory mounted as a volume to the database container. In this article we’ll use /tmp/postgres for that. Any other directory would be good too as long as the PostgreSQL Docker container can access it.

First, we’ll create the local directory with appropriate permissions. This may require root privileges on your system.

$ cd /tmp
$ mkdir postgres
$ chown postgres:postgres postgres/

Having that, let’s mount the new directory as a volume to the PostgreSQL container. For that we’ll add the option -v /tmp/postgres:/tmp/postgres:Z to the Docker run command – for details refer to the guide on running PostgreSQL with Docker. This maps /tmp/postgres from the Docker container to our locally created /tmp/postgres directory.

Note we make use of the “:Z” option for the mounted volume here to overcome potential issues with SELinux. You might not need that depending on you SELinux configuration. For more details also refer to the corresponding section in our MongoDB docker guide.

The final command to launch PostgreSQL with local storage (recommended) and the mounted data transfer directory for COPY’ing data is:

$ docker run -d \
  --name mypostgres \
  --user 1002 \
  -e POSTGRES_PASSWORD=Test123$ \
  -e PGDATA=/var/lib/postgresql/data/pgdata \
  -v /var/db/postgres:/var/lib/postgresql/data:Z \
  -v /tmp/postgres:/tmp/postgres:Z \
  -p 5432:5432 \
  postgres:16.2

That’s it – you now have a PostgreSQL database running in Docker with directory /tmp/postgres mounted to localhost, ready for transferring data.

Extract data to localhost with COPY TO

First, we’ll use COPY to transfer data from a PostgreSQL table to localhost. For that we connect to the database running with Docker and create a simple table tasks having two columns. Then we insert some rows and use COPY to extract the data to a local CSV file.

$ psql -h localhost -p 5432 -U postgres          
Password for user postgres: 

postgres=# CREATE TABLE tasks (id bigint PRIMARY KEY, description varchar(100));
CREATE TABLE
postgres=# INSERT INTO tasks VALUES (1, 'My first Task');
INSERT 0 1
postgres=# INSERT INTO tasks VALUES (2, 'Task Nr. 2');
INSERT 0 1
postgres=# COPY tasks TO '/tmp/postgres/export.csv' DELIMITER ',' CSV HEADER;
COPY 2
postgres=#

Back on localhost after logging out from PostgreSQL we can verify that the data has been written to export.csv in the directory /tmp/postgres of the local file system.

postgres=# quit
$ cat /tmp/postgres/export.csv 
id,description
1,My first Task
2,Task Nr. 2

Brilliant, that’s working as expected.

Insert data from localhost with COPY FROM

Next, we’ll go the other way round and insert data from a local CSV file into a table. For that we place a file import.csv in the transfer directory on localhost.

$ cat /tmp/postgres/import.csv 
id,description
100,Task-100
200,Task-200
300,Task-300

Having that, we connect back again to our PostgreSQL database, truncate the tasks table and populate it with the data from the import file using the COPY command.

$ postgres psql -h localhost -p 5432 -U postgres
Password for user postgres: 

postgres=# TRUNCATE TABLE tasks;
TRUNCATE TABLE
postgres=# COPY tasks FROM '/tmp/postgres/import.csv' DELIMITER ',' CSV HEADER;
COPY 3
postgres=# SELECT * FROM tasks;
 id  | description 
-----+-------------
 100 | Task-100
 200 | Task-200
 300 | Task-300
(3 rows)

postgres=# 

That’s already it. We’ve successfully moved data between the local file system and our dockerized PostgreSQL back and forth using the COPY command.

Useful links

]]>
Running PostgreSQL with Docker on Linux using local persistent data storage https://tsmx.net/postgresql-docker-local-persistent-storage/ Tue, 28 May 2024 20:05:09 +0000 https://tsmx.net/?p=2745 Read more]]> A quick guide demonstrating how to get PostgreSQL up & running in minutes under Linux using Docker. We’ll create local persistent data storage using a Docker volume and connect from localhost with psql.

Preparing local data storage

By default, running PostgreSQL with Docker would store all the data within the container, meaning it would not survive a rebuild of the container. To overcome that, let’s create a local directory that will be mounted as a Docker volume where PostgreSQL can save all of the data to be persistent regardless of container rebuilds etc.

As a best practice, let’s first create a new user called postgres to manage the permissions for all local directories mounted as a volume by the PostgreSQL Docker container. To do so, run the following commands with root privileges.

$ useradd -M postgres
$ usermod -L postgres

This creates a new user postgres having no home directory and a locked password preventing unwanted use. Next, we create a directory (I prefer something under /var/db) for the PostgreSQL data volume and assign it to the newly created user.

$ cd /var/db
$ mkdir postgres
$ chown -R postgres:postgres /var/db/postgres

That’s it for the local preparation. Now let’s move on to start PostgreSQL with Docker. All following steps can be done without having root privileges.

Running PostgreSQL with Docker

To run PostgreSQL with docker, we’ll first download the offical Postgres docker image.

$ docker pull postgres:16.2

Before starting the database, we need to figure out the user ID of the postgres user for passing it to Docker. This will ensure the running container has sufficient rights to access the local data directory mounted as a volume.

$ id -u postgres                                
1002

Now we are ready to run Postgres with Docker using the following command:

$ docker run -d \
  --name mypostgres \
  --user 1002 \
  -e POSTGRES_PASSWORD=Test123$ \
  -e PGDATA=/var/lib/postgresql/data/pgdata \
  -v /var/db/postgres:/var/lib/postgresql/data:Z \
  -p 5432:5432 \
  postgres:16.2

Let’s break down the options of the docker run command in detail.

docker run optionDescription
-dRun the container in background mode.
--name mypostgres[optional] Name the container mypostgres.
--user 1002Set the user ID the container is running with to ensure appropriate rights for accessing the mounted volumes.
-e POSTGRES_PASSWORD=Test123$Set the environment variable for the Postgres admin password.
-e PGDATA=/var/lib/postgresql/data/pgdataSet the environment variable for the Postgres data directory.
-v /var/db/postgres:/var/lib/postgresql/data:ZMount local directory /var/db/postgres as a volume to container directory /var/lib/postgresql/data for persistent data storage on your localhost.
-p 5432:5432Map the Postgres container port 5432 to local port 5432.
postgres:16.2The container image to run.

Note here that the PGDATA variable must be set to something different then /var/lib/postgresql to support mounting to a local persistent directory. For details refere to the corresponding Postgres docker image documentation.

Also you may need the “:Z” option for the volume mounts depending on your SELinux configuration. For an in-depth explanation refer to the Docker MongoDB guide.

Now let’s check if the container is up & running.

$ docker ps | grep postgres
b020097a3a6f   postgres:16.2         "docker-entrypoint.sā€¦"   2 days ago      Up 9 seconds    0.0.0.0:5432->5432/tcp, :::5432->5432/tcp   mypostgres

Looks good – we’re now ready to connect and use the PostgreSQL Docker instance.

Connecting from localhost with psql

Having the container running, let’s connect with the standard Postgres CLI tool psql. If you haven’t installed it already, you should opt-in for a local installation. This is done by installing the postgresl package using dnf with admin rights. If you are on another Linux distribution than Fedora, use your respective package manager.

$ dnf install postgresql

After psql is installed, run the following command as a normal user to connect to the Postgres container from localhost.

$ psql -h localhost -p 5432 -U postgres
Password for user postgres:  # enter the password specified in the docker run command

postgres=#

That’s already it – you’re connected to PostgreSQL running with Docker šŸ™‚

If you want to use other database systems with Docker under Linux, also check out the guides for MongoDB and Oracle.

Useful links

]]>
Running Oracle DB in Docker using the official container image – in minutes, free edition available https://tsmx.net/running-oracledb-in-docker/ Fri, 10 Nov 2023 22:55:11 +0000 https://tsmx.net/?p=2579 Read more]]> Want to run an Oracle database for developing or experimenting for free with minimal setup effort? This article shows how you can get up & running an Oracle DB using Docker in a few minutes.

Searching around for running an Oracle DB on Docker you’ll likely be navigated to the Oracle Database on Docker site on GitHub. This provides comprehensive guides and Dockerfiles to built your own images… interesting, but unfortunately not a quick and simple solution to pull & run a database. So let’s explore a more easy way…

Pulling the official Oracle DB container image

Oracle also provides pre-built Docker images in their own registry located at container-registry.oracle.com.

Navigating to the Database section, you can find images for various versions of the Oracle database like Enterprise, Express or Free. In this article we’ll use the Free version.

By choosing a repository you’ll get additional information on how to pull the image and what parameters are offered for customizing.

oracle-container-registry-database

To use those Docker images, you’ll need an Oracle account to log on to the container registry. If you don’t have one, go ahead and create one for free.

Then log on to the Oracle container registry and pull the database image, e.g. the latest free edition.

$ docker login container-registry.oracle.com

$ docker pull container-registry.oracle.com/database/free:latest

$ docker image ls | grep oracle                                                                                                                
container-registry.oracle.com/database/free    latest    39cabc8e6db0    2 months ago    9.16GB

Having this, you are ready to start the database.

Starting the Oracle database with Docker

Before we start a new container running the database, its recommendable to create a local directory where all the data files can persist outside the container. By doing so, the data persistence is decoupled from the containers lifeycycle. In our example the local directory /opt/oracle/oradata was created with write permissions for any user.

Now you can start up the Oracle DB like so:

$ docker run --name oracle \                                                                                                                   
-p 1521:1521 \
-e ORACLE_PWD=Test123 \
-v /opt/oracle/oradata:/opt/oracle/oradata \
container-registry.oracle.com/database/free:latest

This will start a container named oracle binding it to port 1521 and setting the SYS password of the database to Test123. The local directory /opt/oracle/oradata will be mounted directly in the container ensuring that database files are persistent.

Note 1: If you encounter any permission problems with the data directory under Linux, any SELinux settings are likely the cause. To overcome this, a quick solution is to pass the :Z option to the volume parameter:
-v /opt/oracle/oradata:/opt/oracle/oradata:Z
For more details on that, just have a look on the article about moving MongoDB to Docker.

Note 2: On first startup when the mounted oradata directory is empty, Oracle will create a completely new database in there which might take some minutes.

After the startup is completed, you should see DATABASE IS READY TO USE in the output. Also this gives you the SID of the started DB which is FREE in that case.

Awesome, now let’s connect and use the database…

Connecting to the database

After startup is completed you can connect to the database at localhost:1521 as SYS using the supplied password Test123.

Using SQLDeveloper…

…or using sqlplus

$ sqlplus sys/Test123@localhost as sysdba

SQL*Plus: Release 21.0.0.0.0 - Production on Fri Nov 10 21:08:06 2023
Version 21.12.0.0.0

Copyright (c) 1982, 2022, Oracle.  All rights reserved.


Connected to:
Oracle Database 23c Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.3.0.23.09

SQL> 

…or any other DB tool you want.

That’s already it. Have fun playing around with Oracle DB šŸ™‚

Useful links

]]>
Moving MongoDB from local installation to Docker – Fedora migration guide https://tsmx.net/local-mongodb-to-docker-fedora-migration-guide/ Wed, 23 Nov 2022 22:12:56 +0000 https://tsmx.net/?p=1830 Read more]]> Complete guide for moving an existing local MongoDB installation to newer versions of Fedora using Docker. Also useful if you are looking for getting a MongoDB container up & running on Linux with locally hosted data.

Problems with a traditional MongoDB installation on Fedora

Trying to install a MongoDB community server locally on a newer version of Fedora (in my case MongoDB 5 on Fedora 36) could be a mess. In the good old days using a repo with DNF was the best option…

$ cat /etc/yum.repos.d/mongodb-org-5.0.repo
[mongodb-org-5.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/8/mongodb-org/5.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-5.0.asc
$ dnf install mongodb-org
Last metadata expiration check: 0:10:51 ago on Tue Nov 22 21:14:41 2022.
Error: 
 Problem: conflicting requests
  ...
  - nothing provides /usr/libexec/platform-python needed by mongodb-org-database-tools-extra-5.0.0-1.el8.x86_64
  (try to add '--skip-broken' to skip uninstallable packages)

Since needed dependencies have been dropped in newer Fedora releases (platform-python at least), this doesn’t work any more out-of-the-box. Falling back to a manual rpm install, this also is not working straight forward since MongoDB is relying on older crypto libraries not shipped with current Fedora releases…

$ rpm -ihv mongodb-org-server-5.0.14-1.el8.x86_64.rpm 
warning: mongodb-org-server-5.0.14-1.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID e2c63c11: NOKEY
error: Failed dependencies:
        libcrypto.so.1.1()(64bit) is needed by mongodb-org-server-5.0.14-1.el8.x86_64
        libcrypto.so.1.1(OPENSSL_1_1_0)(64bit) is needed by mongodb-org-server-5.0.14-1.el8.x86_64
        libssl.so.1.1()(64bit) is needed by mongodb-org-server-5.0.14-1.el8.x86_64
        libssl.so.1.1(OPENSSL_1_1_0)(64bit) is needed by mongodb-org-server-5.0.14-1.el8.x86_64
        libssl.so.1.1(OPENSSL_1_1_1)(64bit) is needed by mongodb-org-server-5.0.14-1.el8.x86_64 

You could now investigate hours on how to solve this problems manually, but at the end of the day keep in mind that MongoDB also clearly states that they don’t provide package support for Fedora. So this is a good starting point to think about a more stable & standard alternative, which could be running MongoDB as a Docker container…

Migration goals

Assuming you already have a locally running MongoDB in an older version of Fedora (or any other Linux distribution), this guide focusses on how to:

  • Running MongoDB community server as a Docker container on Fedora or another Linux system
  • Migrating all data and permissions (users/groups) to the Docker version without the need of dumping or other heavy lifting
  • Showing how MongoDB’s data, meta data and configuration can be held outside the Docker container to gain maximum flexibility

Moving from local MongoDB installation to dockerized version

Step 1: Backup existing MongoDB data and configuration

First we’ll need to locate and save all of the current data.

  • Database data: Locate the path where MongoDB is storing all of its data. Normally this is /var/lib/mongo on a Fedora system or another dbPath provided at startup via the --dbPath option or in the storage section of the configuration file which usually is /etc/mongod.conf.
    After you have located the path, backup it entirely including subfolders (e.g. copy to a USB Stick, your NAS or whatever).
  • Configuration data: If your current MongoDB is using a configuration file – normally located at /etc/mongod.conf – or another file provided via the --config parameter then also save this file.

Step 2: Prepare the new environment

To run MongoDB using docker with data and configuration hosted outside the container for flexibility reasons we should first create an appropriate place for that. To do so, switch to root and create a new user mongod and two directories data and conf under /var/db to separate all from existing users and home directories.

$ useradd -M mongod
$ usermod -L mongod
$ cd /var/db
$ mkdir mongo
$ mkdir mongo/data
$ mkdir mongo/conf

The created user mongod has no home directory and is also not permitted to log-in using an password as both is not needed in our scenario. Having this in place, copy the saved MongoDB data from step 1 to the new directories. Finally, change the ownership of all files to the mongod user.

Assuming you have stored the data on an USB stick mounted as /run/myuser/usbstick, like so:

$ cp -R /run/myuser/usbstick/mongo/data /var/db/mongo/data
$ cp /run/myuser/usbstick/mongo/conf /var/db/mongo/conf
$ chown -R mongod:mongod /var/db/mongo

Note: For executing the next steps, you can switch back from root to your normal user.

Step 3: Run MongoDB as a Docker container

First, we should get the appropriate offical image from Docker Hub. To avoid version incompatibilities I recommend start using the same version of MongoDB you had in your local installation before. In my case it was 5.0, so I choosed the image tagged with 5.0.13.

$ docker pull mongo:5.0.13

After downloading, the last thing to do before starting it up is to figure out the id of the mongod user which is the owner of all data in the filesystem.

$ id -u mongod
1005

Now we can start up the MongoDB container with docker run like so:

$ docker run \
-p 27017:27017 \
--user 1005 \ 
-v /var/db/mongo/data:/data/db \
-v /var/db/mongo/conf:/etc/mongo \
mongo:5.0.13 \
--config /etc/mongo/mongod.conf 
...
{
  "t":{"$date":"2022-11-23T20:02:25.878+00:00"},
  "s":"I",
  "c":"NETWORK",
  "id":23016,   
  "ctx":"listener",
  "msg":"Waiting for connections",
  "attr":{"port":27017,"ssl":"off"}
}

Awesome! MongoDB is now running and waiting for connections on port 27017. All databases, collections, users & roles are there like they were before in the local installation.

Note that this runs MongoDB directly in the current shell. To stop it, simply press CTRL-C. If you rather want it to run in the background, add the -d option to the docker run command.

For a better understanding, let’s break down the command issued above.

Command expressionExplanation
docker runTell Docker to start a new container. See the docker run docs for all details.
-p 27017:27017Exposes MongoDB’s standard port 27017 from the container to your local system. Enables you to connect via localhost:27017 to the MongoDB container like it was with a local installation.
--user 1005Tell Docker to run the container as user with id 1005 (= mongod). This is essential since the MongoDB image will depend on that the owner of the local data/config files are identical. Otherwise it’ll try to chown them which would fail.
-v /var/db/mongo/data:/data/dbMaps the local directory /var/db/mongo/data to /data/db in the container. This is the MongoDB image default place for database data inside the container. Can be changed with the dbPath parameter in the storage section of the config file.
-v /var/db/mongo/conf:/etc/mongoMaps the local directory /var/db/mongo/conf to /etc/mongo in the container. This is the MongoDB image default place for the configuration file inside the container. Can be changed with the --config parameter – see below.
mongo:5.0.13Tell Docker to use the image mongo with tag 5.0.13 when running the container.
--config /etc/mongo/mongod.confTell MongoDB to use the specified configuration file within the container.

Troubleshooting

I cannot connect to the MongoDB container also it started up without any error

If everything seems to be running fine without any errors but you are still not able to connect to MongoDB via localhost:27017 it is most likely that your configuration prohibits it.

In that case, please check the configuration in mongod.conf (or the file you use) under /var/db/mongo/conf and there the IP’s allowed to bind to MongoDB. Allowing 0.0.0.0 should solve it and be sufficient in the Docker scenario.

# network interfaces
net:
  port: 27017
  bindIp: 0.0.0.0

The container won’t start because of denied permissions when acessing the local filesystem

If you try to start up the MongoDB container like described above and get an error saying that permissions on the filesystem are denied like so…

dpkg: warning: failed to open configuration file '/data/db/.dpkg.cfg' for reading: Permission denied
{
  "t":{"$date":"2022-11-23T21:09:43.425Z"},
  "s":"F",  
  "c":"CONTROL",  
  "id":6384300, 
  "ctx":"-",
  "msg":"Writing fatal message",
  "attr":{"message":"terminate() called. An exception is active; attempting to gather more information\n"}
}
{
  "t":{"$date":"2022-11-23T21:09:43.425Z"},
  "s":"F",  
  "c":"CONTROL",  
  "id":6384300, 
  "ctx":"-",
  "msg":"Writing fatal message",
  "attr":{"message":"std::exception::what(): boost::filesystem::status: Permission denied: \"/etc/mongo/mongod.conf\"\nActual exception type: boost::filesystem::filesystem_error\n\n"}
}

Then you should first ensure that the ownership of the local filesystem is correctly set to the mongod user. For that, execute as root:

$ chown -R mongod:mongod /var/db/mongo

If that doesn’t solve the problem, it is very likely that you have SELinux enabled with an enforcing policy which prohibits docker from accessing the files. You can check the status with sestatus.

$ sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      33

In simple words, the Current mode: enforcing is telling you that SELinux is running in a “strict-mode” where you explicitly have to allow Docker to access the local filesystem.

There are three options to solve this:

1. Set SELinux to permissive mode by executing setenforce 0 as root. Note that this change is temporary and has to be re-executed after a reboot.

$ setenforce 0
$ sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   permissive
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      33

2. Modify the docker run command an add the :Z option (please note it is a capital Z) to the volume mappings, like so.

$ docker run \
-p 27017:27017 \
--user 1005 \ 
-v /var/db/mongo/data:/data/db:Z \
-v /var/db/mongo/conf:/etc/mongo:Z \
mongo:5.0.13 \
--config /etc/mongo/mongod.conf 

3. Relabel the needed directories and files to set the right SELinux context for Docker as decsribed in this very good article.

Option 3 “relabeling” should be the best solution, but to be honest since it is a very complex process I personally decided to go for the :Z addition in the volume mappings and didn’t investigate the relabeling further. In the Docker docs it is stated to better not use this option with /home and /usr directories, which in fact we are not doing here.

Have a great time with your dockerized MongoDB šŸ™‚

Useful links

]]>
Connecting from a Docker container to a local MongoDB https://tsmx.net/docker-local-mongodb/ Fri, 14 Jan 2022 22:18:03 +0000 https://tsmx.net/?p=1484 Read more]]> A quick guide demonstrating how to connect to a local MongoDB instance from a Docker container.

Docker containers run in an isolated environment including separate virtual networks by default. Although you can run your MongoDB in a container connected to such a virtual network, e.g. by using docker-compose, there may be situations where you want to use a local MongoDB instance and connect to it from a container.

The Docker bridge network

To establish networking connections between the host and containers, a Docker bridge network can be used. A standard one called bridge is always present and generated by default. You can inspect it using the docker network inspect bridge command.

$ docker network inspect bridge
[
    {
        "Name": "bridge",
        "Id": "73cb0c87fb64f19c76e3367e80b2da2d104a67603882ca539ce2cf70bdf97c4f",
        "Created": "2022-01-14T20:53:38.577610832+01:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        },
        "Labels": {}
    }
]

Notice that this network has a Gateway with the IP 172.17.0.1. This we will use to connect from the container to our MongoDB running on the host.

According to the Docker docs on networking the default bridge network should not be used for production scenarios but is fine for any dev/test environment. In this article we will continue using the default bridge.

Connecting to your local MongoDB from Docker

For connecting to your local MongoDB instance from a Container you must first allow to accept connections from the Docker bridge gateway. To do so, simply add the respective gateway IP in the MongoDB config file /etc/mongod.conf under bindIp in the network interface section.

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1,172.17.0.1

Note that binding to 0.0.0.0 would also do the trick but is not recommended due to security reasons.

Second, you should choose a hostname to access the database. Let’s use mongoservice for that. In your containerized app, use this hostname in the connection string, like so…

mongodb://mongoservice:27017/mydb

To introduce the mongoservice hostname in the containers virtual network and bind it to our local host where MongoDB is running, simply use the --add-host option of docker run with the bridge networks gateway IP we discovered earlier.

docker run --add-host=mongoservice:172.17.0.1 repository/image-name

Having this in place, your Docker container now can communicate with the host using the alias mongoservice and your are good to go in connecting to the local MongoDB šŸ™‚

Tip: using the identical hostname alias for MongoDB locally

To avoid inconsistencies and configuration changes between running your app in the local dev environment and the Docker container, simply add an entry for the used hostname mongoservice to your /etc/hosts.

127.0.0.1   localhost localhost.localdomain
::1         localhost localhost.localdomain
127.0.0.1   mongoservice

Now you can use the exactly same connection string also in your local dev environment.

Troubleshooting: MongoDB isn’t starting up any more

After adding the Docker bridge gateway IP to MongoDB’s configuration like described before you may face an issue when trying to start MongoDB. Although everything was fine before.

$ systemctl start mongod
Job for mongod.service failed because the control process exited with error code.
See "systemctl status mongod.service" and "journalctl -xeu mongod.service" for details.
$ systemctl status mongod
Ɨ mongod.service - MongoDB Database Server
     Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Thu 2022-03-17 21:08:18 CET; 36s ago
       Docs: https://docs.mongodb.org/manual
    Process: 16629 ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb (code=exited, status=0/SUCCESS)
    Process: 16630 ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb (code=exited, status=0/SUCCESS)
    Process: 16631 ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb (code=exited, status=0/SUCCESS)
    Process: 16632 ExecStart=/usr/bin/mongod $OPTIONS (code=exited, status=48)

Mar 17 21:08:18 fedora systemd[1]: Starting MongoDB Database Server...
Mar 17 21:08:18 fedora mongod[16632]: about to fork child process, waiting until server is ready for connections.
Mar 17 21:08:18 fedora mongod[16634]: forked process: 16634
Mar 17 21:08:18 fedora mongod[16632]: ERROR: child process failed, exited with 48
Mar 17 21:08:18 fedora mongod[16632]: To see additional information in this output, start without the "--fork" option.
Mar 17 21:08:18 fedora systemd[1]: mongod.service: Control process exited, code=exited, status=48/n/a
Mar 17 21:08:18 fedora systemd[1]: mongod.service: Failed with result 'exit-code'.
Mar 17 21:08:18 fedora systemd[1]: Failed to start MongoDB Database Server.

If this is the case, it is very likely that Docker is not running! Make sure to start Docker before MongoDB. Otherwise MongoDB will fail to start because the Docker bridge gateway isn’t available and it cannot bind to it.

$ systemctl start docker
$ systemctl start mongod

If you want to start MongoDB without having Docker running, then you’ll need to remove or comment out the gateway’s IP in /etc/mongod.conf.

Happy coding šŸ™‚

Useful links

]]>