Could not resolve placeholder 'spring.datasource.url' in value "${spring.datasource.url}"

[Jf Okeeffe]

Hi,
I received an application to work on which I was not the one who developed it.
Now I'm getting this SpringBoot error message.

...Could not resolve placeholder 'spring.datasource.url' in value "${spring.datasource.url}"

From what I investigated, this value should be in the application.properties file.
But the application doesn't have that file. At least it is not in the repository.

So, do you know of any other place that this value could be or is the application.properties file really missing and needs to be created?

[Tim Holloway]

There is a very long list of places where a Spring Boot app can get its application properties from, including which sources take priority over others. I cannot recall the exact URL, but it's in the Spring Boot docs.

In addition to application properties as a Java Properties file, Spring Boot also supports application properties in YML-format files.

Also, you can supply/override application properties from an external config file. I feed database connection parameters into one of my apps that way.

[Himai Minh]

Hello Jf,
In your IDE , under your project, do you have resource folder? If so, create application.properties file in resource folder.

[Jf Okeeffe]

Himai Minh wrote:Hello Jf,
In your IDE , under your project, do you have resource folder? If so, create application.properties file in resource folder.

Yes, I could create the application.properties file.
But the point is that I'd like to understand how the application was actually built because seems it doesn't have that file and it is up and running in a prod server.
So my goal is to reproduce the way it works now.
I don't have access to the prod server to see if the file is there though.

[Tim Holloway]

Jf Okeeffe wrote:

Himai Minh wrote:Hello Jf,
In your IDE , under your project, do you have resource folder? If so, create application.properties file in resource folder.

Yes, I could create the application.properties file.
But the point is that I'd like to understand how the application was actually built because seems it doesn't have that file and it is up and running in a prod server.
So my goal is to reproduce the way it works now.
I don't have access to the prod server to see if the file is there though.

For stuff like database properties, it's definitely something you'd want in an external config file. Ideally developers would never know the necessary connection parameters for a production database server.

Though to facilitate development and testing, it's not a bad idea to have overrideable properties within the project itself pointing to the test database. And in any case, a heads-up in the project's README file!!!

[Tim Holloway]

This might help. It's what the demo for my recipe manager webapp (https://gourmetj.mousetech.com) uses to define both the database connection info and the location of the userid/password file. Note that this is the YML format, so the indented levels concatenate to form the non-YML (properties) names. So I do have a "spring.datasource.url" defined there. No userid/password, since this is a SQLite database and has no security parameters. For the latest source version, I switched to MySQL so the usual extra parameters apply as well.

This file is located in the deployed app directory /META-INF/resources/application.yml file, although other external locations are also possible.
# # Application YML is used in production. /application.properties # is used in test. # spring:  datasource:    url:  jdbc:sqlite:${home}/recipes.db  jpa:    hibernate:      ddl-auto: none    database-platform: org.sqlite.hibernate.dialect.SQLiteDialect gourmet:    password:        file: .gourmetpw

[Jf Okeeffe]

Hi Tim,
There are no yml files in the project. So that's not how it is set.
I searched for "datasource" using IntelliJ's "Find in Files" feature.
I could find some files that have a javax.sql.DataSource object.
But still can't find where the database connection data is being passed to them.

[Jf Okeeffe]

Also could not find any occurence of @PropertySource which could be the place where a different properties file is set.

[Tim Holloway]

javax.sql.Datasource is normally created from a JNDI lookup of a JDBC Connection Pool created by the Tomcat server. Since this is embedded Tomcat, that pool's configuration will be injected into the Tomcat bean from the application properties. You might want to search the project for "jdbc" and see if you can find more info in the search results.

Please note that YAML isn't the only way to do this, I just prefer it over flat properties files. The important thing is that for the particular app I illustrated, the database info isn't in the app, it's in the production server as file external to the Spring Boot JAR.

[Ron McLeod]

Jf Okeeffe wrote:Could not resolve placeholder 'spring.datasource.url' in value "${spring.datasource.url}"
       ....
So, do you know of any other place that this value could be or is the application.properties file really missing and needs to be created?

I haven't worked with SpringBoot before, but I assume that it could also pick-up properties like this from environment variables.  Is SPRING_DATASOURCE_URL defined anywhere on the target environment?

[Tim Holloway]

Ron McLeod wrote:

Jf Okeeffe wrote:Could not resolve placeholder 'spring.datasource.url' in value "${spring.datasource.url}"
       ....
So, do you know of any other place that this value could be or is the application.properties file really missing and needs to be created?

I haven't worked with SpringBoot before, but I assume that it could also pick-up properties like this from environment variables.  Is SPRING_DATASOURCE_URL defined anywhere on the target environment?

That wouldn't work anyway. Wrong case, wrong punctuation. And no, I wouldn't do that anyway, since it requires a more complex startup sequence with no extra benefit. Simpler to just put in an external properties file.

Here's a little help, but I'm still looking for the official Spring Boot docs on it:

https://www.tutorialspoint.com/spring_boot/spring_boot_application_properties.htm

[Tim Holloway]

Ah, this should be it:

https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.external-config.files

[Ron McLeod]

Tim Holloway wrote:Wrong case, wrong punctuation.

Replacement of special characters with underscores is required for environments which don't allow periods and other characters in env variables names (like the various Linux shells).  Uppercasing of names also common.

From the document that you shared:
7.2.3. External Application Properties: If you use environment variables rather than system properties, most operating systems disallow period-separated key names, but you can use underscores instead (for example, SPRING_CONFIG_NAME instead of spring.config.name).

[Tim Holloway]

Ron McLeod wrote:

Tim Holloway wrote:Wrong case, wrong punctuation.

Replacement of special characters with underscores is required for environments which don't allow periods and other characters in env variables names (like the various Linux shells).  Uppercasing of names also common.

From the document that you shared:
7.2.3. External Application Properties: If you use environment variables rather than system properties, most operating systems disallow period-separated key names, but you can use underscores instead (for example, SPRING_CONFIG_NAME instead of spring.config.name).

Regardless, it's last on my list. A '-D' definition would be preferable, and a permanent config file more commonly (since it doesn't expose sensititive data in a process status display).

[Jf Okeeffe]

Tim Holloway wrote:You might want to search the project for "jdbc" and see if you can find more info in the search results.

Yes. There is this.
It's using Hikari.
Now will need to investigate where spring.datasource.url is set in Hikari.

	@Value("${spring.datasource.url}")
	private String url;
	...
	@Bean
	public DataSource dataSource() {	
		...		
		HikariConfig config = new HikariConfig();
		config.setJdbcUrl(url);
		...
	}

[Tim Holloway]

I think maybe you're wasting time here. Consider setting up an external config file like the sample one I showed you and test with that. Whatever Production has done works for them, so you shouldn't have to be concerned about how they do it.

It's almost certainly NOT done in the project source code, based on what you've found, and really, it shouldn't be. It isn't for me, even. Production database connection information shouldn't be something seen outside of Production.

[Jf Okeeffe]

Tim Holloway wrote:Production database connection information shouldn't be something seen outside of Production.

Well, having access to the production db connection data is not what I'm looking for.
But I need to know how the application connects to the db in prod and replicate that in my local dev envioronment using a dev db.
That's what I'm looking for here.
Because, at some point I'll need to update the repo and prod with my changes and don't intend to change how that db connection process works.

[Tim Holloway]

That's my point. If, as it should be, the production connection is already set up and operational, then when you submit the updated app to production, it should hook in automatically. If it does not, then someone other than you has failed to properly document procedures. In which case, they'd need to roll back, run around, scream, point fingers and at some point actually resolve the issue.

There are, as the docs I listed show, many many ways to inject the JDBC connection parameters into a Spring Boot app and it really isn't your responsibility to know which method Production uses it or even duplicate it exactly, since they all have the same net effect. If I use an XML userid/password Realm file for my Tomcat test app, it makes no difference whether the production server actually uses a JDBC or LDAP credentials provider. It's externally-injected and not my job.

Well, in my case, it sometimes is, but that's because long before "DevOps" became a Thing, I was senior enough to get called into the computer room if things went really sour. But I kept it to a minimum even then so that someone else could get yelled at and almost never actually touched the code or data myself, just made suggestions.

Again, a well-run IT shop will maintain production secrets — including database URL/userid/password information — solely in Production. It's their problem, not yours.

[Jf Okeeffe]

Tim Holloway wrote:If, as it should be, the production connection is already set up and operational, then when you submit the updated app to production, it should hook in automatically.

Sure, I understand your point. Although, in my case, I'd rather know how things work in prod to be prepared in case anything goes wrong during a prod update, instead of just expecting it will work automatically.

Regardless, I'd say that whether going with one approach or the other, the technical issue I initially raised is still valid and not solved. So would prefer to focus the discussion from that perspective. Even if it was just for a learning purpose.

Thanks for the Spring Boot link. I'm looking into that and Hikari.

[Tim Holloway]

Understandable and commendable. It really should be part of the system documentation, so the Production people should be able (and I hope! willing) to tell you so that it can be formally documented. If you have DevOps handling deployment, they should know.

Because there are so many different valid ways to inject that information, your best result for panic repairs would be to know what they are currently using. If that isn't possible, of course, you can always override at a higher priority level.

For testing, use whatever suits you.