When we need to spin up a database instance for our new project, installing the database management system directly on our local machine is almost always a bad idea. Luckily, Docker is here to help us practically reduce the complexity of dealing with "missing dependencies" and weird error messages down to zero. Show In this lesson, we will learn how to get a PostgreSQL instance running locally with Docker Compose, so we can work on the database integration of our project and test things out with ease. To follow this lesson, you will need to have Docker pre-installed. ✍️ Week 1 lesson 5 of DataTalksClub 2022 data engineering zoomcamp, using 🎼 Docker Compose to build 🕸️ network with 🐘 Postgres and pgAdmin Today, we will follow DataTalksClub's video: DE Zoomcamp 1.2.5 - Running Postgres and pgAdmin with Docker-Compose. Which is part of the DataTalksClub 2022 Data engineering Zoomcamp week 1 repo. In our last post, we learned how to convert a data ingestion notebook into a Python script. Then, we added the script to a container specifying that it should execute when the container runs, following DataTalksClub's video: DE Zoomcamp 1.2.4 - Dockerizing the Ingestion Script. 🗳️ Dockerizing a data ingestion script ✍️ Week 1 lesson 4 of DataTalksClub 2022 data engineering zoomcamp, turn notebook 📓 into script 📜, and run it with Docker 🗳️ pintonistaRafaelAs you saw in the previous posts, a lot of configuration takes place in the terminal to build each docker container and the network. For cases like this, a better approach is to use Docker Compose, where we create one YML file with all the containers' configurations. 💬 In this lesson, we will:
This post is part of a series. Find all the other related posts here DataTalkClub 2022 data engineering zoomcamp ✍️ Table of contents for my learning in public posts on DataTalksClub 2022 data engineering zoomcamp pintonistaRafael💻 Installing Docker ComposeWe will follow the Linux systems installation instructions in the Docker docs to install Docker Compose. First, we download and install it with
and then, we add executable permissions to the binary with
✏️ Create a Docker Compose fileLet's peek into the to see the typical file components. Now, create a docker-compose.yamlWhen we run Docker Compose using this file, we will be able to access the database with the service name specified here ( 0) from pgAdmin. Also, since we defined both services here, they automatically become part of the same network.🏃♀️ Running Docker ComposeFirst, we must stop the containers we started in our last post (Postgres and pgAdmin) by going to their respective locked terminal window and typing 1. We can check if the containers are running with the 2 command. 🛑 Running containers that must be stopped. The empty table returned after stopping containers.To run Docker Compose, go to a terminal window and change directories until you reach our working directory (the one with the
which will run the containers specified in the Running Docker Compose.We can now go to a web browser and refresh the pgAdmin tab ( 5), log in with the credentials specified in the docker-compose.yaml file and configure the server again, as shown below. To get to the server configuration screen, MB1 on the Server tab on pgAdmin left pane, then MB2 over it, and follow Create > Server...Creating a Server in pgAdmin.Fill in the server name.Fill in the hostname, username, and password.After completing the server configuration, MB1 on the Save button. Now we can check if the data is still there by using pgAdmin's query tool. If you need a refresher on how to do this, we wrote the details in our . 🛑 Stopping docker-composeTypically, we will start docker-compose in detached mode with
so the terminal doesn't get locked by the running process. Since we didn't start it with this option, we must exit the process with 1 and then stop the containers with
📝 SummaryIn this post we:
In our next lesson, we will flex our SQL muscle 💪 by loading the taxi zones lookup table and reviewing JOIN and GROUP BY operations. 💽 SQL refresher ✍️ Week 1 lesson 6 of DataTalksClub 2022 data engineering zoomcamp, reviewing 💽 SQL basics with the 🚕 NYC taxi trips data pintonistaRafael#Course Notes: 2022 DataTalksClub Data Engineering Zoomcamp #Learning in public #Data Engineering #Docker How to install Postgres using docker compose?Procedure. Navigate to the bwce_mon directory.. Run the following command to build the application monitoring image. ... . Run the following command, which downloads the PostgreSQL image and configures the database with admin user and bwcemon database. ... . Run the following command to start the monitoring server on 8080 port.. How do I add PgAdmin to docker compose?Run PostgreSQL and pgAdmin in docker for local development using docker compose. create a directory. mkdir postgres-docker && cd postgres-docker.. create a file and name it as docker-compose.yml.. Add the following content in the docker-compose,yml file.. How to run PostgreSQL in docker container?To launch your Postgres database and supporting services, enter the docker compose -f [FILE NAME] up command. Using either docker run , psql , or Docker Compose, you can successfully start up Postgres using the Official Image! These are reliable ways to work with “default” Postgres.
What is the default user for Postgres docker compose?The default user is 'Postgres'. Port – We need to define on which port is the database going to run in the container and which port on the host can be used to access it. As you can see, there are two ports mentioned in the command which are in the form HOST_PORT:DOCKER_PORT.
|