Django: dynamic database file

Question

In my Django project I have a dependency for a third party application that produces SQLite cache files in various directories with a known schema.

I'd like to use Django models to access those databases, but obviously I cannot use a static DATABASES setup.

How can I dynamically open a SQLite database on an arbitrary path?

EDIT

As Byron Ruth pointed out, the solution is to use the django.db.connections in conjunction with the using function in the QuerySet.

why cannot use DATABASES? From what I see this why the authors of Django made it! The other option is to use raw sql (http://docs.python.org/2/library/sqlite3.html), but kinda defeats the purpose of a framework! — StefanNch, Jan 10 '13 at 09:21
@StefanNch: I cannot use the `DATABASES` setup because I cannot know what databases I'll end up with, because they are added *dynamically*. And yes, I'm currently using the sqlite3 API, which I'm not fully content with as I do not want to dig into SQL. — Constantinius, Jan 10 '13 at 09:24
thats an interesting architecture, but now I see your issue ... — StefanNch, Jan 10 '13 at 09:41

score 33 · Accepted Answer · edited May 04 '15 at 13:54

33

The django.db.connections is a simple wrapper around DATABASES defined in your settings. The wrapper class is here: django.db.utils#L137-L227

from django.db import connections

# Add connection information dynamically..
connections.databases['new-alias'] = { ... }
# Ensure the remaining default connection information is defined.
# EDIT: this is actually performed for you in the wrapper class __getitem__
# method.. although it may be good to do it when being initially setup to
# prevent runtime errors later.
# connections.databases.ensure_defaults('new-alias')

# Use the new connection
conn = connections['new-alias']

edited May 04 '15 at 13:54

Gautam

7,868
12
64
105

answered Jan 21 '13 at 16:06

Byron Ruth

927
6
15

1

I think this is the solution I was looking for. I'll give it a try! – Constantinius Jan 22 '13 at 08:35
1

This was it, I was able to dynamically select a database file, open it and use my models. (this requires the `using` function on the queryset, though). – Constantinius Jan 22 '13 at 08:39
nice! hmmm... so this connection needs to be added for each incoming request that needs to use the db? Might be nice to wrap this in some middleware to setup the db as the request comes in... – monkut Nov 14 '13 at 02:59
2

@monkut they will be stored for the duration of the process. a quick `if 'new-alias' not in connections.databases` check would ensure the settings are always present. On a side note, if you _really_ want a temporary database connection try this: https://gist.github.com/bruth/7467130 – Byron Ruth Nov 14 '13 at 13:57
This method also doesn't appear to allow `model_instance.save(using='new-alias'). Is there a way this functionality can be added? – monkut Nov 20 '13 at 16:26
where we have to put this ? For example I am using constance then how can i configure this ? – Deepak Tripathi Oct 27 '21 at 05:53

score 4 · Answer 2 · answered Jan 10 '13 at 09:24

4

You can register database in DATABASES settings.

from your_project import settings
database_id = "unqique_name"
new_database = {}
new_database["id"] = database_id
new_database['ENGINE'] = 'django.db.backends.sqlite3'
new_database['NAME'] = '/project/data/db_%s.sql' % database_id
new_database['USER'] = ''
new_database['PASSWORD'] = ''
new_database['HOST'] = ''
new_database['PORT'] = ''
settings.DATABASES[database_id] = new_database

You can but you shouldn't.

answered Jan 10 '13 at 09:24

baklarz2048

10,699
2
31
37

This looks interesting, I'll give it a try. But I'd prefer a solution where I don't have to alter the `DATABASES` setting. – Constantinius Jan 10 '13 at 09:28
2

You shouldn't alter settings in your applications at runtime. For example, don't do this in a view: https://docs.djangoproject.com/en/dev/topics/settings/#altering-settings-at-runtime – Adem Öztaş Jan 10 '13 at 10:01

score 1 · Answer 3 · answered Jan 20 '13 at 01:05

1

Assuming the only engine used is SQLite and the location of the (only) database file is what varies, provide a callable to the NAME:

def get_db_loc():
    # code to determine filesystem location of database
    return location

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': get_db_loc(),
        # More config goes here
    }
}

answered Jan 20 '13 at 01:05

stellarchariot

2,872
1
19
28

I would second this. If the DB locations change dynamically during runtime, make sure that get_db_loc() changes its behavior based on which DB is currently active (store that info in a text file/global var), and then inside get_db_loc() in real time redirect to the currently active DB. – Jan 21 '13 at 22:08
2

I'm not sure this will work, because the function `get_db_loc` is only evaluated once, and not every time the `DATABASES` is read. But otherwise thanks for the contribution. – Constantinius Jan 22 '13 at 08:29
Right, there is no way this could work. The result of `get_db_loc()` is just a string. There's nothing special about the fact that it was obtained by calling a function, and there's nothing that says to ever call that function again. – rspeer Nov 10 '14 at 22:33
Ah, fair enough. Changing from a function invocation to a callback might work? `get_db_loc()` becomes `get_db_loc` *shrugs* – stellarchariot Nov 13 '14 at 06:28

Django: dynamic database file

3 Answers3

Linked