SymfonyCasts stands united with the people of Ukraine
gstreamer0.10-ffmpeg
gstreamer0.10-plugins-good packages.
We need to get a database running: MySQL, Postgresql, whatever. If you already have one running, awesome! All you need to do is copy your DATABASE_URL environment variable, open or create a .env.local file, paste, then change it to match whatever your local setup is using. If you decide to do this, feel free to skip ahead to the end of chapter 4 where we configure the server_version.
For me, I do not have a database running locally on my system... and I'm not going to install one. Instead, I want to use Docker. And, we're going to use Docker in an interesting way. I do have PHP installed locally:
php -v
So I won't use Docker to create a container specifically for PHP. Instead I'm going to use Docker simply to help boot up any services my app needs locally. And right now, I need a database service. Thanks to some magic between Docker and the Symfony binary, this is going to be super easy.
To start, remember when the Doctrine recipe asked us if we wanted Docker configuration? Because we said yes, the recipe gave us docker-compose.yml and docker-compose.override.yml files. When Docker boots, it will read both of these... and they're split into two pieces just in case you want to also use Docker to deploy to production. But we're not going to worry about that: we just want to use Docker to make life easier for local development.
| version: '3' | |
| services: | |
| ###> doctrine/doctrine-bundle ### | |
| database: | |
| image: postgres:${POSTGRES_VERSION:-13}-alpine | |
| environment: | |
| POSTGRES_DB: ${POSTGRES_DB:-app} | |
| # You should definitely change the password in production | |
| POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-ChangeMe} | |
| POSTGRES_USER: ${POSTGRES_USER:-symfony} | |
| volumes: | |
| - db-data:/var/lib/postgresql/data:rw | |
| ... lines 14 - 22 |
| version: '3' | |
| services: | |
| ###> doctrine/doctrine-bundle ### | |
| database: | |
| ports: | |
| - "5432" | |
| ... lines 8 - 9 |
These files say that they will boot a single Postgres database container with a user called symfony and password ChangeMe:
Tip
The username changed from symfony to app in the newest recipe version.
It will also expose port 5432 of the container - that's Postgres's normal port - to our host machine on a random port. This means that we're going to be able to talk to the Postgresql Docker container as if it were running on our local machine... as long as we know the random port that Docker chose. We'll see how that works in a minute.
By the way, if you want to use MySQL instead of Postgres, you absolutely can. Feel free to update these files... or delete both of them and run:
php bin/console make:docker:database
to generate a new compose file for MySQL or MariaDB. I'm going to stick with Postgres because it's awesome.
At this point, we're going to start Docker and learn a bit about how to communicate with the database that lives inside. If you're pretty comfortable with Docker, feel free to skip to the next chapter.
Anyways, let's get our container running. First, make sure you have Docker actually installed on your machine: I won't show that because it varies by operating system. Then, find your terminal and run:
docker-compose up -d
The -d means "run in the background as a daemon". The first time you run this, it'll probably download a bunch of stuff. But eventually, our container should start!
Cool! But now what? How can we talk to the container? Run a command called:
docker-compose ps
This shows info about all the containers currently running... just one for us. The really important thing is that port 5432 in the container is connected to port 50700 on my host machine. This means that if we talk to this port, we will actually be talking to that Postgres database. Oh, and this port is random: it'll be different on your machine... and it'll even change each time we stop and start our container. More on that soon.
But now that we know about port 50700, we can use that to connect to the database. For example, because I'm using Postgres, I could run:
psql --user=symfony --port=50700 --host=127.0.0.1 --password app
That means: connect to Postgres at 127.0.0.1 port 50700 using user symfony and talking to the app database. All of this is configured in the docker-compose.yml file. Copy the ChangeMe password because that last flag tells Postgres to ask for that password. Paste and... we're in!
If you're using MySQL, we can do this same thing with a mysql command.
But, this only works if we have that psql command installed on our local machine. So let's try a different command. Run:
docker-compose ps
again. The container is called database, which comes from our docker-compose.yml file. So we can change the previous command to:
docker-compose exec database psql --username symfony --password app
This time, we're executing the psql command inside the container, so we don't need to install it locally. Type ChangeMe for the password and... we're back in!
The point is: just by running docker-compose up, we have a Postgres database container that we can talk to!
Btw, when you're ready to stop the container later, you can run:
docker-compose stop
That basically turns the container off. Or you can run the more common:
docker-compose down
which turns off the containers and removes them. To start back up, it's the same:
docker-compose up -d
But notice that when we run docker-compose ps again, the port on my host machine is a different random port! So, in theory, we could configure the DATABASE_URL variable to point to our Postgres database, including using the correct port. But that random port that keeps changing is going to be annoying!
Fortunately, there's a trick for this! It turns our, our app is already configured, without us doing anything! That's next.
by the way, symfony console doctrine:database:create gives the following error:
An exception occurred in the driver: SQLSTATE[08006] [7] connection to server at "127.0.0.1", port 49154 failed: FATAL: password authentication failed for user "app"
Hi there!
Sorry for the trouble, but I think I know what the problem is. Since a few weeks ago, the Doctrine recipe comes with Pgsql 14 instead of 13. That's actually fine. However, it's possible that you have an existing volume from running Docker earlier that had a pgsql "data" directory for pgsql 13. And this is causing problem with pgsql 14 operating properly.
This is what happened to me, and it gave me the same error that you're all getting. To see the error, try to start docker without the -d:
docker-compose up -d
This gave me a very clear error:
The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 14.4.
So, where did this "volume" come from? With the docker-compose.yml file, when you start a database, it creates a volume whose name is generated from the name of the directory you're inside. If you download the course code and move into the start directory, then your directory name is start. This will cause a volume called start_db-data to be created, which stores your database data.
You can see all your volumes by running:
docker volume ls
So, if you're like me, the problem is caused by using Docker with pgsql 13 previously inside a directory named start. What's the fix? There are two:
1) Delete the old volume - this will only work if that docker container is no longer running: docker volume rm start_db-data
2) Rename the directory you're inside of.
Let me know if that helps!
Cheers!
That's unfortunately not my problem. My version is 14 on both, but it somehow did not make a role:
spellencafe-database-1 | 2022-12-11 12:46:06.167 UTC [40] FATAL: password authentication failed for user "=app"
spellencafe-database-1 | 2022-12-11 12:46:06.167 UTC [40] DETAIL: Role "=app" does not exist.
spellencafe-database-1 | Connection matched pg_hba.conf line 100: "host all all all scram-sha-256"
Now, when I log in with docker-compose exec and try and CREATE ROLE app it says:
[58] ERROR: role "app" already exists
I`m guessing this has nothing to do with Symfony but more so with either PostgreSQL and/or Docker. If you could point me in the right direction, I would be greatfull.
No clue what's happening, but I downgraded the Postgres_version in docker-compose.yml from 14 to 13 (since that's the version on my production server). Although my local psql is on 14 and the server is now on 13, it seems to work, so far...
Hello @weaverryan
I do not get the same error as your's when I run docker-compose up
Here is what I get :
➜ mixed_vinyl git:(main) ✗ docker-compose up
Creating network "mixed_vinyl_default" with the default driver
Creating mixed_vinyl_database_1 ... done
Attaching to mixed_vinyl_database_1
database_1 |
database_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
database_1 |
database_1 | 2022-12-08 10:56:33.092 UTC [1] LOG: starting PostgreSQL 14.6 on aarch64-unknown-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
database_1 | 2022-12-08 10:56:33.092 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
database_1 | 2022-12-08 10:56:33.092 UTC [1] LOG: listening on IPv6 address "::", port 5432
database_1 | 2022-12-08 10:56:33.094 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
database_1 | 2022-12-08 10:56:33.096 UTC [21] LOG: database system was shut down at 2022-12-07 16:04:40 UTC
database_1 | 2022-12-08 10:56:33.101 UTC [1] LOG: database system is ready to accept connections
database_1 | 2022-12-08 10:59:56.159 UTC [34] FATAL: password authentication failed for user "app"
database_1 | 2022-12-08 10:59:56.159 UTC [34] DETAIL: Connection matched pg_hba.conf line 100: "host all all all scram-sha-256"
seems to be logging so we can see the failed connexion attempt
bye the way, here is the content of the services key in my docker-compose.yml :
database:
image: postgres:${POSTGRES_VERSION:-14}-alpine
environment:
POSTGRES_DB: ${POSTGRES_DB:-app}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-ChangeMe}
POSTGRES_USER: ${POSTGRES_USER:-app}
volumes:
- db-data:/var/lib/postgresql/data:rw
Hmm, does renaming your directory to something else help? I'm still suspect that there is an existing volume using pgsql 13 that is causing problems. Also, I can't think of why this would make a difference, but oddly your POSTGRES_PASSWORD is slightly different than the one that comes with the recipe: https://github.com/symfony/recipes/blob/main/doctrine/doctrine-bundle/2.4/manifest.json#L31
Cheers!
Hello @weaverryan.
I just started a new symfony project with symfony new and run the composer require doctrine commande right after, to put your theory to the test.
In this new project, everything seems to work perfectly :
I can connect to the database in the container with psql --user=app --port=56899 --host=127.0.0.1 --password app
and the symfony console doctrine:database:create yells at me something about a database "app" thas already exists... ;)
I'm glad to start this new week with Symfony, especially with this problem solved.
Thanks a lot for your help, and for all these useful (and fun) videos.
You rock, have a nice week.
PS : renaming the folder of the mixed_vinyl project works, too
Hey Xesh! Thank you for the update! This tells me that, at least some people had the same issue as me :).
Have a great week!
Def
| posted 22 days ago
Hello, I am running my symfony local web server with this integration docker. However, even though my docker-compose services are running, I can not get symfony to connect to them.
The debug toolbar says Docker Compose Down under the sf Server menu.
From what I see I don't get any errors regarding this
Any thoughts? If there is some more information I can provide please let me know.
Thanks.
Hey Def,
Hoe do you run your Symfony app? You should not use bin/console calls anymore if you're going to use Docker, you would need to replace all your commands with symfony console instead - that will handle all the Docker integration for you :)
So, e.g. instead of bin/console doctrine:query:execute run symfony console doctrine:query:execute, etc.
If still does not work - try to "down" all your containers. i.e. run docker composer down and then up them again with docker composer up - there might be a misconfiguration etc.
Cheers!
Thx Victor for your help
Unfortunatly, it seems like it's a biggest issue with my docker conf because i tried to install a whole new symfony project with a new docker-compose configuration and when i start my local webserver on this new project and up my containers I also have a docker compose down and "Env vars" none, so... yeah
I don't know what to do but thanks
Hey Def,
Hm, not sure then, I don't use Docker daily :) Maybe try to go through Docker docs to reinstall and configure it from scratch, maybe something is missing. Actually, check for updates first, maybe you just need to upgrade to the latest. Btw, laptop restart may help sometimes too.
Cheers!
Stanislaw
| posted 1 month ago
Just quick note from my journey: if you change the password in .env file and docker-compose.yml before running docker and creating container for the first time, it's all good, but if you changed it later you will have an error password authentication failed for user... if you want to use new password you have to basically delete container and create new one, but you can also return to old password, which you used during creation of db.
Hey Stanislaw,
Thanks for this tip! Yeah, you need to run docker compose down to drop all the created containers and start them again with docker compose up.
Cheers!
Juergen-S
| posted 6 months ago
You have to add your user to group "docker". Otherwise you will run into an error.
See: https://stackoverflow.com/questions/64662372/docker-compose-up-error-while-fetching-server-api-version-connection-abortesudo gpasswd -a $USER docker
Hey,
Thanks for the tip. Can you please share your OS version? Probably it's important because we all use different systems =)
Cheers and happy coding!
EricSod
| posted 8 months ago
Hi. I'm getting invalid compose project
➜ docker-compose up -d
service "database" refers to undefined volume db-data: invalid compose project
These are the docker apps I'm running.
➜ docker -v && docker-compose -v
Docker version 20.10.21, build baeda1f82a
Docker Compose version 2.14.1
I'm using the code downloaded from this page, renamed the start directory to mixed_vinyl
If I change db-data on line 13 of docker-compose.yml to database, the error message is updated
➜ docker-compose up -d
service "database" refers to undefined volume database: invalid compose project
symfony serve -d serves https://127.0.0.1:8000/ without a problem.
Any thoughts? Thanks.
Hey Eric!
Apologies for my very slow reply! Holidays started early this year at my house :p.
Ok, this sounds frustrating :/. So, I CAN repeat your error message, but only if I change docker-compose.yml in a specific way. Here is an abbreviated version of my docker-compose.yml file, which came (like in the tutorial) from the DoctrineBundle recipe after installing Doctrine:
services:
###> doctrine/doctrine-bundle ###
database:
# ...
volumes:
- db-data:/var/lib/postgresql/data:rw
###< doctrine/doctrine-bundle ###
volumes:
###> doctrine/doctrine-bundle ###
db-data:
###< doctrine/doctrine-bundle ###
With this config (well, again, I'm hiding less-important parts, but hopefully you get the idea), things work. How can I repeat your error? By removing the volumes section. In other words, it seem like, in your config, the volumes section is missing... or perhaps it's called something different than db-data. Basically, because we have - db-data:... above, we should have a volume called db-data: near the bottom.
Let me know if that helps! For reference, here is my full, unchanged, docker-compose.yml file after installing Doctrine (this version works for me):
version: '3'
services:
###> doctrine/doctrine-bundle ###
database:
image: postgres:${POSTGRES_VERSION:-14}-alpine
environment:
POSTGRES_DB: ${POSTGRES_DB:-app}
# You should definitely change the password in production
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-!ChangeMe!}
POSTGRES_USER: ${POSTGRES_USER:-app}
volumes:
- db-data:/var/lib/postgresql/data:rw
# You may use a bind-mounted host directory instead, so that it is harder to accidentally remove the volume and lose all your data!
# - ./docker/db/data:/var/lib/postgresql/data:rw
###< doctrine/doctrine-bundle ###
volumes:
###> doctrine/doctrine-bundle ###
db-data:
###< doctrine/doctrine-bundle ###
Cheers!
Hi Ryan,
Yes, having volumes: with a key that matches the volumes: in services.database.volumes helped. But my problem was I've been using colima as a docker context. I usually run a .ddev/colima with mutagen stack, which works great for Drupal and avoids the subscription service for Docker desktop as well as the Docker desktop app.
Once, I changed the docker context back to default (which means docker) and started the docker desktop app (bleh), the docker daemon was started and the symfony app found the database through doctrine right away.
My next questions are out of scope, but do you know how to run the docker daemon without the docker desktop app?
And can Colima be used as a replacement for Docker in Symfony?
Thanks. Enjoy the rest of the holidays.
Hey EricSod!
Ah, ok, this makes more sense then!
Once, I changed the docker context back to default (which means docker) and started the docker desktop app (bleh), the docker daemon was started and the symfony app found the database through doctrine right away.
Tbh, I'm not familiar with Colima, though it looks super interesting. Unfortunately, because I'm a bit clueless about it (other than reading about it just now for about 10 minutes), I don't know why you had the volumes problem when using it. But, as far as Symfony + Docker + Database is concerned the volumes part isn't important. What's important is that your docker-compose.yaml has a service called database. So, the whole solution may still be workable, but I don't know enough to say how.
My next questions are out of scope, but do you know how to run the docker daemon without the docker desktop app?
I'm a bit clueless here too :). But, from my reading, it sounds like one of the points of colima is to be able to run Docker without needing Docker desktop - but just by running brew install docker? I have a feeling you already knew that, but let me know.
And can Colima be used as a replacement for Docker in Symfony?
I can't say for certain, but the symfony basically looks for a docker-compose.yaml file, loops over the services inside, then (using the name of each service) it creates some environment variables that point to that service. If you satisfy both of these requirements - regardless of it's all setup, then it should all work. But also, keep in mind that this setup is just an "easy" way to get a database setup in Docker and hooked up with your Symfony app. It should be making your life easier, not harder. If it's not doing that, then don't use it (and that's totally ok!).
Cheers!
With the docker desktop app running, set the docker default to colima with
docker context use colima
and run
docker-compose up -d
The prompt returns
Cannot connect to the Docker daemon at unix:///Users/esod/.colima/default/docker.sock. Is the docker daemon running?
Symfony profiler reports Server -> Docker Compose -> Down.
Switch the docker default to docker with
docker context use default
run docker-compose up -d and confirm the Symfony profiler is reporting Docker Compose -> Up.
Note: Quit the Docker Desktop app and Symfony profiler reports Server -> Docker Compose -> Down.
Sounds like two problems now. Yay DevOps. :-)
Anyway, the recipe for success is what you're recommending in the tutorial.
Hey EricSod!
Thanks for the detailed reply - and boo when things don't work more easily! :P Happy you're at least not blocked.
Cheers!
Laurent-G
| posted 8 months ago
Hello, i have a problem when i run this commandpsql --user=app --port=52439 --host=127.0.0.1 --password app, the result is :
psql: The term 'psql' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
Docker is running on Windows 11, if i executedocker-compose ps, the result is :
NAME COMMAND SERVICE STATUS PORTS
mixed_vinyl-database-1 "docker-entrypoint.s…" database running 0.0.0.0:52439->5432/tcp
What is doing to resolve this mistake ?
Thanks,
Laurent
Hey Laurent-G,
It seems like you're missing that psql command installed locally, though the error message is a bit weird to me. Do you have psql installed locally? You should have it installed to be able to connect to the DB from Docker. On my Mac when I execute psql --version it shows me psql (PostgreSQL) 14.5 (Homebrew), IIRC I installed the PostgreSQL via Brew package manager locally and it brought me that psql command. If you don't have PostgreSQL installed locally - try to install it first. But brew is specific for Mac, you should find a way to install PostgreSQL on Windows.
Of if you don't want to (or cannot) install it locally - see the next chapter where we execute that psql command directly from the Docker container.
Cheers!
Thanks Victor for your answer.
i don't have installed psql locally. I'm going to see for install psql on my windows system.
i already saw in the next chapter the command to execute psql from Docker container and it's work very good.
Thanks a lot,
Laurent
Hey Laurent-G,
Glad that executing it from the Docker container works for you, thanks for confirming! I bet you just need to install PostgreSQL and it will bring that psql command into your system automatically. Or you can just ignore it and execute it directly from the Docker container - that's the benefit of having Docker ;)
Cheers!
red_smeg
| posted 9 months ago
Is there a version of this with a mysql container. Tried using Postgres's but getting data from an existing mysql database to the Postgres container is too hard
figured it out (lost a lot of time because of the stupid ! at the start of the !ChangeMe! password......)
services:
database:
container_name: database
image: mysql:8.0
command: --default-authentication-plugin=mysql_native_password
environment:
MYSQL_ROOT_PASSWORD: yoursecretrootpassword
MYSQL_DATABASE: yourdbname
MYSQL_USER: app
MYSQL_PASSWORD: yourpasswd <-- the one you used in the .env file
ports:
- '3306:3306'
volumes:
- ./mysql:/var/lib/mysql`
Hey Red,
I'm happy to see you were able to find the solution! And thanks for sharing it with others :)
Cheers!
Markchicobaby
| posted 9 months ago
Hi, running command "docker-compose exec database psql --username symfony --password app" I'm prompted for a password, which I enter (to match the one in the docker compose file which is !ChangeMe! but then... it just hangs. Cursor flashes, but no response at all. I can connect to the database successfully using DBeaver (a GUI database client), it connects and can see the database fine. This is what I see at the terminal:
`$ docker-compose exec database psql --username app --password app
Password: !ChangeMe!
`
Hey Mark,
Hm, could you change that "!ChangeMe!" password in .env files to something without special chars, I guess that the exclamation char (!) may cause issues. And then restart the server and try to connect with the new credentials.
Cheers!
Hi and thanks for this new tutorial!
I've an error when I run docker-compose exec database psql --username symfony --password app:psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "symfony" does not exist
Am I missing something?
OS: Linux Mint 20.3 Cinnamon 5.2.7
Kernel: 5.4.0-124-generic
Docker version 20.10.17, build 100c70
docker-compose version 1.28.6, build 5db8d86f
Hey Thomas A.!
Oh sheesh - try this instead - changing "symfony" to "app":
docker-compose exec database psql --username app --password app
Explanation: RIGHT after recording, the recipe was updated - you probably have the new version https://github.com/symfony/recipes/commit/b3395a2477b6d58089f92ef3a0d2e58226825a3a#diff-b52cb1d9a056076027dd6033aa8c573524746ed54fcdeeb5c8eef83431b9da9aL32
Let me know if that helps - we'll need to add a note about it :).
Cheers!
// composer.json
{
"require": {
"php": ">=8.1",
"ext-ctype": "*",
"ext-iconv": "*",
"babdev/pagerfanta-bundle": "^3.7", // v3.7.0
"doctrine/doctrine-bundle": "^2.7", // 2.7.0
"doctrine/doctrine-migrations-bundle": "^3.2", // 3.2.2
"doctrine/orm": "^2.12", // 2.12.3
"knplabs/knp-time-bundle": "^1.18", // v1.19.0
"pagerfanta/doctrine-orm-adapter": "^3.6", // v3.6.1
"pagerfanta/twig": "^3.6", // v3.6.1
"sensio/framework-extra-bundle": "^6.2", // v6.2.6
"stof/doctrine-extensions-bundle": "^1.7", // v1.7.0
"symfony/asset": "6.1.*", // v6.1.0
"symfony/console": "6.1.*", // v6.1.2
"symfony/dotenv": "6.1.*", // v6.1.0
"symfony/flex": "^2", // v2.2.2
"symfony/framework-bundle": "6.1.*", // v6.1.2
"symfony/http-client": "6.1.*", // v6.1.2
"symfony/monolog-bundle": "^3.0", // v3.8.0
"symfony/proxy-manager-bridge": "6.1.*", // v6.1.0
"symfony/runtime": "6.1.*", // v6.1.1
"symfony/twig-bundle": "6.1.*", // v6.1.1
"symfony/ux-turbo": "^2.0", // v2.3.0
"symfony/webpack-encore-bundle": "^1.13", // v1.15.1
"symfony/yaml": "6.1.*", // v6.1.2
"twig/extra-bundle": "^2.12|^3.0", // v3.4.0
"twig/twig": "^2.12|^3.0" // v3.4.1
},
"require-dev": {
"doctrine/doctrine-fixtures-bundle": "^3.4", // 3.4.2
"symfony/debug-bundle": "6.1.*", // v6.1.0
"symfony/maker-bundle": "^1.41", // v1.44.0
"symfony/stopwatch": "6.1.*", // v6.1.0
"symfony/web-profiler-bundle": "6.1.*", // v6.1.2
"zenstruck/foundry": "^1.21" // v1.21.0
}
}
I unfortunately also have a problem connecting. In docker-compose.yml a have the following setup:
Now, when I run the command
psql --user=symfony --port=49153 --host=127.0.0.1 --password appand give the password, I get the error:However, when I use the command
docker-compose exec database psql --user app --password appand give the password, I do get in which makes no sense to me. Could you please demystify?