Python - Get path of root project structure
I've got a python project with a configuration file in the project root.
The configuration file needs to be accessed in a few different files throughout the project.
So it looks something like: <ROOT>/configuration.conf
<ROOT>/A/a.py
, <ROOT>/A/B/b.py
(when b,a.py access the configuration file).
What's the best / easiest way to get the path to the project root and the configuration file without depending on which file inside the project I'm in? i.e without using ../../
? It's okay to assume that we know the project root's name.
python
|
show 2 more comments
I've got a python project with a configuration file in the project root.
The configuration file needs to be accessed in a few different files throughout the project.
So it looks something like: <ROOT>/configuration.conf
<ROOT>/A/a.py
, <ROOT>/A/B/b.py
(when b,a.py access the configuration file).
What's the best / easiest way to get the path to the project root and the configuration file without depending on which file inside the project I'm in? i.e without using ../../
? It's okay to assume that we know the project root's name.
python
does<ROOT>/__init__.py
exist?
– mgilson
Aug 19 '14 at 17:08
Either your configuration file is a python module, and you can easily access it just with an import statement, either it's not a python module and you should put it in a well known location. For example $HOME/.my_project/my_project.conf.
– John Smith Optional
Aug 19 '14 at 17:09
@JohnSmithOptional - It's a JSON file. I need to be able to access it using the path. Yes. All of the folders include it.
– Shookie
Aug 19 '14 at 17:09
_ It's okay to assume that we know the project root's name._ Does that mean you know the path to the project? Isn't it just os.path.join(known_root_name, "configuration.conf") then?
– tdelaney
Aug 19 '14 at 17:19
If it's a user configuration I'd generally use something likeos.path.expanduser('~/.myproject/myproject.conf')
. It works on Unix and Windows.
– John Smith Optional
Aug 19 '14 at 17:22
|
show 2 more comments
I've got a python project with a configuration file in the project root.
The configuration file needs to be accessed in a few different files throughout the project.
So it looks something like: <ROOT>/configuration.conf
<ROOT>/A/a.py
, <ROOT>/A/B/b.py
(when b,a.py access the configuration file).
What's the best / easiest way to get the path to the project root and the configuration file without depending on which file inside the project I'm in? i.e without using ../../
? It's okay to assume that we know the project root's name.
python
I've got a python project with a configuration file in the project root.
The configuration file needs to be accessed in a few different files throughout the project.
So it looks something like: <ROOT>/configuration.conf
<ROOT>/A/a.py
, <ROOT>/A/B/b.py
(when b,a.py access the configuration file).
What's the best / easiest way to get the path to the project root and the configuration file without depending on which file inside the project I'm in? i.e without using ../../
? It's okay to assume that we know the project root's name.
python
python
asked Aug 19 '14 at 17:02
ShookieShookie
1,76242345
1,76242345
does<ROOT>/__init__.py
exist?
– mgilson
Aug 19 '14 at 17:08
Either your configuration file is a python module, and you can easily access it just with an import statement, either it's not a python module and you should put it in a well known location. For example $HOME/.my_project/my_project.conf.
– John Smith Optional
Aug 19 '14 at 17:09
@JohnSmithOptional - It's a JSON file. I need to be able to access it using the path. Yes. All of the folders include it.
– Shookie
Aug 19 '14 at 17:09
_ It's okay to assume that we know the project root's name._ Does that mean you know the path to the project? Isn't it just os.path.join(known_root_name, "configuration.conf") then?
– tdelaney
Aug 19 '14 at 17:19
If it's a user configuration I'd generally use something likeos.path.expanduser('~/.myproject/myproject.conf')
. It works on Unix and Windows.
– John Smith Optional
Aug 19 '14 at 17:22
|
show 2 more comments
does<ROOT>/__init__.py
exist?
– mgilson
Aug 19 '14 at 17:08
Either your configuration file is a python module, and you can easily access it just with an import statement, either it's not a python module and you should put it in a well known location. For example $HOME/.my_project/my_project.conf.
– John Smith Optional
Aug 19 '14 at 17:09
@JohnSmithOptional - It's a JSON file. I need to be able to access it using the path. Yes. All of the folders include it.
– Shookie
Aug 19 '14 at 17:09
_ It's okay to assume that we know the project root's name._ Does that mean you know the path to the project? Isn't it just os.path.join(known_root_name, "configuration.conf") then?
– tdelaney
Aug 19 '14 at 17:19
If it's a user configuration I'd generally use something likeos.path.expanduser('~/.myproject/myproject.conf')
. It works on Unix and Windows.
– John Smith Optional
Aug 19 '14 at 17:22
does
<ROOT>/__init__.py
exist?– mgilson
Aug 19 '14 at 17:08
does
<ROOT>/__init__.py
exist?– mgilson
Aug 19 '14 at 17:08
Either your configuration file is a python module, and you can easily access it just with an import statement, either it's not a python module and you should put it in a well known location. For example $HOME/.my_project/my_project.conf.
– John Smith Optional
Aug 19 '14 at 17:09
Either your configuration file is a python module, and you can easily access it just with an import statement, either it's not a python module and you should put it in a well known location. For example $HOME/.my_project/my_project.conf.
– John Smith Optional
Aug 19 '14 at 17:09
@JohnSmithOptional - It's a JSON file. I need to be able to access it using the path. Yes. All of the folders include it.
– Shookie
Aug 19 '14 at 17:09
@JohnSmithOptional - It's a JSON file. I need to be able to access it using the path. Yes. All of the folders include it.
– Shookie
Aug 19 '14 at 17:09
_ It's okay to assume that we know the project root's name._ Does that mean you know the path to the project? Isn't it just os.path.join(known_root_name, "configuration.conf") then?
– tdelaney
Aug 19 '14 at 17:19
_ It's okay to assume that we know the project root's name._ Does that mean you know the path to the project? Isn't it just os.path.join(known_root_name, "configuration.conf") then?
– tdelaney
Aug 19 '14 at 17:19
If it's a user configuration I'd generally use something like
os.path.expanduser('~/.myproject/myproject.conf')
. It works on Unix and Windows.– John Smith Optional
Aug 19 '14 at 17:22
If it's a user configuration I'd generally use something like
os.path.expanduser('~/.myproject/myproject.conf')
. It works on Unix and Windows.– John Smith Optional
Aug 19 '14 at 17:22
|
show 2 more comments
8 Answers
8
active
oldest
votes
You can do this how Django does it: define a variable to the Project Root from a file that is in the top-level of the project. For example, if this is what your project structure looks like:
project/
configuration.conf
definitions.py
main.py
utils.py
In definitions.py
you can define (this requires import os
):
ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) # This is your Project Root
Thus, with the Project Root known, you can create a variable that points to the location of the configuration (this can be defined anywhere, but a logical place would be to put it in a location where constants are defined - e.g. definitions.py
):
CONFIG_PATH = os.path.join(ROOT_DIR, 'configuration.conf') # requires `import os`
Then, you can easily access the constant (in any of the other files) with the import statement (e.g. in utils.py
): from definitions import CONFIG_PATH
.
To include the definitions.py file like that, will it be required to add a__init__.py
file to the root project directory as well ? Should that be correct ? I've just started with python and not sure on the best practices. Thanks.
– akskap
Aug 30 '16 at 8:20
1
@akskap: No, an__init__.py
will not be required, as that file is only required when defining packages: The__init__.py
files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case,__init__.py
can just be an empty file, but it can also execute initialization code for the package or set the__all__
variable, described later. See: docs.python.org/3/tutorial/modules.html#packages
– jrd1
Aug 31 '16 at 6:02
I am curious, style wise, whether it is acceptable or frowned upon to add these definitions to the__init.py__
of the root package. It would save creating another file, as well as allow the nicer syntax offrom root_pack import ROOT_DIR, CONFIG_PATH
.
– Johndt6
Jan 5 '17 at 2:30
@Johndt6: the convention is to keep__init__.py
empty, but that isn't strictly true (it's a convention after all). See this for more: stackoverflow.com/questions/2361124/using-init-py
– jrd1
Jan 5 '17 at 5:58
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:47
|
show 5 more comments
To get the path of the "root" module, you can use:
import os
import sys
os.path.dirname(sys.modules['__main__'].__file__)
But more interestingly if you have an config "object" in your top-most module you could -read- from it like so:
app = sys.modules['__main__']
stuff = app.config.somefunc()
1
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:50
add a comment |
Other answers advice to use file in top-level of the project. This is not necessary if you use pathlib.Path
and parent
. Consider the following directory structure where all files except README.md
and utils.py
have been omitted.
project
│ README.md
|
└───src
│ │ utils.py
| | ...
| ...
In utils.py
we define the following function.
from pathlib import Path
def get_project_root() -> Path:
"""Returns project root folder."""
return Path(__file__).parent.parent
In any module in the project we can now get the project root as follows.
from src.utils import get_project_root
root = get_project_root()
Benefits: Any module which calls get_project_root
can be moved without changing program behavior. Only when the module utils.py
is moved we have to update get_project_root
and the imports (use IDE refactoring to automate this).
add a comment |
A standard way to achieve this would be to use the pkg_resources
module which is part of the setuptools
package. setuptools
is used to create an install-able python package.
You can use pkg_resources
to return the contents of your desired file as a string and you can use pkg_resources
to get the actual path of the desired file on your system.
Let's say that you have a package called stackoverflow
.
stackoverflow/
|-- app
| `-- __init__.py
`-- resources
|-- bands
| |-- Dream Theater
| |-- __init__.py
| |-- King's X
| |-- Megadeth
| `-- Rush
`-- __init__.py
3 directories, 7 files
Now let's say that you want to access the file Rush from a module app.run
. Use pkg_resources.resouces_filename
to get the path to Rush and pkg_resources.resource_string
to get the contents of Rush; thusly:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('resources.bands', 'Rush')
print pkg_resources.resource_string('resources.bands', 'Rush')
The output:
/home/sri/workspace/stackoverflow/resources/bands/Rush
Base: Geddy Lee
Vocals: Geddy Lee
Guitar: Alex Lifeson
Drums: Neil Peart
This works for all packages in your python path. So if you want to know where lxml.etree
exists on your system:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('lxml', 'etree')
output:
/usr/lib64/python2.7/site-packages/lxml/etree
The point is that you can use this standard method to access files that are installed on your system (e.g pip install xxx or yum -y install python-xxx) and files that are within the module that you're currently working on.
1
I like your band choice !
– dylan_fan
Jun 20 '18 at 12:06
Ah yes. The good ol' boys. 10 bucks is 10 bucks, eh?
– shrewmouse
Jun 21 '18 at 13:42
add a comment |
All the previous solutions seem to be overly complicated for what I think you need, and often didn't work for me. The following one-line command does what you want:
import os
ROOT_DIR = os.path.abspath(os.curdir)
add a comment |
This worked for me using a standard PyCharm project with my virtual environment (venv) under the project root directory.
Code below isnt the prettiest, but consistently gets the project root. It returns the full directory path to venv from the VIRTUAL_ENV
environment variable e.g. /Users/NAME/documents/PROJECT/venv
It then splits the path at the last /
, giving an array with two elements. The first element will be the project path e.g. /Users/NAME/documents/PROJECT
import os
print(os.path.split(os.environ['VIRTUAL_ENV'])[0])
1
This won't work with setups like anaconda or pipenv, since the virtual environment isn't contained within the project in those cases.
– Gripp
Jul 6 '18 at 11:17
add a comment |
I've recently been trying to do something similar and I have found these answers inadequate for my use cases (a distributed library that needs to detect project root). Mainly I've been battling different environments and platforms, and still haven't found something perfectly universal.
Code local to project
I've seen this example mentioned and used in a few places, Django, etc.
import os
print(os.path.dirname(os.path.abspath(__file__)))
Simple as this is, it only works when the file that the snippet is in is actually part of the project. We do not retrieve the project directory, but instead the snippet's directory
Similarly, the sys.modules approach breaks down when called from outside the entrypoint of the application, specifically I've observed a child thread cannot determine this without relation back to the 'main' module. I've explicitly put the import inside a function to demonstrate an import from a child thread, moving it to top level of app.py would fix it.
app/
|-- config
| `-- __init__.py
| `-- settings.py
`-- app.py
app.py
#!/usr/bin/env python
import threading
def background_setup():
# Explicitly importing this from the context of the child thread
from config import settings
print(settings.ROOT_DIR)
# Spawn a thread to background preparation tasks
t = threading.Thread(target=background_setup)
t.start()
# Do other things during initialization
t.join()
# Ready to take traffic
settings.py
import os
import sys
ROOT_DIR = None
def setup():
global ROOT_DIR
ROOT_DIR = os.path.dirname(sys.modules['__main__'].__file__)
# Do something slow
Running this program produces an attribute error:
>>> import main
>>> Exception in thread Thread-1:
Traceback (most recent call last):
File "C:Python2714libthreading.py", line 801, in __bootstrap_inner
self.run()
File "C:Python2714libthreading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "main.py", line 6, in background_setup
from config import settings
File "configsettings.py", line 34, in <module>
ROOT_DIR = get_root()
File "configsettings.py", line 31, in get_root
return os.path.dirname(sys.modules['__main__'].__file__)
AttributeError: 'module' object has no attribute '__file__'
...hence a threading-based solution
Location independent
Using the same application structure as before but modifying settings.py
import os
import sys
import inspect
import platform
import threading
ROOT_DIR = None
def setup():
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
Breaking this down:
First we want to accurately find the thread ID of the main thread. In Python3.4+ the threading library has threading.main_thread()
however, everybody doesn't use 3.4+ so we search through all threads looking for the main thread save it's ID. If the main thread has already exited, it won't be listed in the threading.enumerate()
. We raise a RuntimeError()
in this case until I find a better solution.
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
Next we find the very first stack frame of the main thread. Using the cPython specific function sys._current_frames()
we get a dictionary of every thread's current stack frame. Then utilizing inspect.getouterframes()
we can retrieve the entire stack for the main thread and the very first frame.
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
Finally, the differences between Windows and Linux implementations of inspect.getouterframes()
need to be handled. Using the cleaned up filename, os.path.abspath()
and os.path.dirname()
clean things up.
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
So far I've tested this on Python2.7 and 3.6 on Windows as well as Python3.4 on WSL
add a comment |
Try:
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f25389095%2fpython-get-path-of-root-project-structure%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
8 Answers
8
active
oldest
votes
8 Answers
8
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can do this how Django does it: define a variable to the Project Root from a file that is in the top-level of the project. For example, if this is what your project structure looks like:
project/
configuration.conf
definitions.py
main.py
utils.py
In definitions.py
you can define (this requires import os
):
ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) # This is your Project Root
Thus, with the Project Root known, you can create a variable that points to the location of the configuration (this can be defined anywhere, but a logical place would be to put it in a location where constants are defined - e.g. definitions.py
):
CONFIG_PATH = os.path.join(ROOT_DIR, 'configuration.conf') # requires `import os`
Then, you can easily access the constant (in any of the other files) with the import statement (e.g. in utils.py
): from definitions import CONFIG_PATH
.
To include the definitions.py file like that, will it be required to add a__init__.py
file to the root project directory as well ? Should that be correct ? I've just started with python and not sure on the best practices. Thanks.
– akskap
Aug 30 '16 at 8:20
1
@akskap: No, an__init__.py
will not be required, as that file is only required when defining packages: The__init__.py
files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case,__init__.py
can just be an empty file, but it can also execute initialization code for the package or set the__all__
variable, described later. See: docs.python.org/3/tutorial/modules.html#packages
– jrd1
Aug 31 '16 at 6:02
I am curious, style wise, whether it is acceptable or frowned upon to add these definitions to the__init.py__
of the root package. It would save creating another file, as well as allow the nicer syntax offrom root_pack import ROOT_DIR, CONFIG_PATH
.
– Johndt6
Jan 5 '17 at 2:30
@Johndt6: the convention is to keep__init__.py
empty, but that isn't strictly true (it's a convention after all). See this for more: stackoverflow.com/questions/2361124/using-init-py
– jrd1
Jan 5 '17 at 5:58
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:47
|
show 5 more comments
You can do this how Django does it: define a variable to the Project Root from a file that is in the top-level of the project. For example, if this is what your project structure looks like:
project/
configuration.conf
definitions.py
main.py
utils.py
In definitions.py
you can define (this requires import os
):
ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) # This is your Project Root
Thus, with the Project Root known, you can create a variable that points to the location of the configuration (this can be defined anywhere, but a logical place would be to put it in a location where constants are defined - e.g. definitions.py
):
CONFIG_PATH = os.path.join(ROOT_DIR, 'configuration.conf') # requires `import os`
Then, you can easily access the constant (in any of the other files) with the import statement (e.g. in utils.py
): from definitions import CONFIG_PATH
.
To include the definitions.py file like that, will it be required to add a__init__.py
file to the root project directory as well ? Should that be correct ? I've just started with python and not sure on the best practices. Thanks.
– akskap
Aug 30 '16 at 8:20
1
@akskap: No, an__init__.py
will not be required, as that file is only required when defining packages: The__init__.py
files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case,__init__.py
can just be an empty file, but it can also execute initialization code for the package or set the__all__
variable, described later. See: docs.python.org/3/tutorial/modules.html#packages
– jrd1
Aug 31 '16 at 6:02
I am curious, style wise, whether it is acceptable or frowned upon to add these definitions to the__init.py__
of the root package. It would save creating another file, as well as allow the nicer syntax offrom root_pack import ROOT_DIR, CONFIG_PATH
.
– Johndt6
Jan 5 '17 at 2:30
@Johndt6: the convention is to keep__init__.py
empty, but that isn't strictly true (it's a convention after all). See this for more: stackoverflow.com/questions/2361124/using-init-py
– jrd1
Jan 5 '17 at 5:58
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:47
|
show 5 more comments
You can do this how Django does it: define a variable to the Project Root from a file that is in the top-level of the project. For example, if this is what your project structure looks like:
project/
configuration.conf
definitions.py
main.py
utils.py
In definitions.py
you can define (this requires import os
):
ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) # This is your Project Root
Thus, with the Project Root known, you can create a variable that points to the location of the configuration (this can be defined anywhere, but a logical place would be to put it in a location where constants are defined - e.g. definitions.py
):
CONFIG_PATH = os.path.join(ROOT_DIR, 'configuration.conf') # requires `import os`
Then, you can easily access the constant (in any of the other files) with the import statement (e.g. in utils.py
): from definitions import CONFIG_PATH
.
You can do this how Django does it: define a variable to the Project Root from a file that is in the top-level of the project. For example, if this is what your project structure looks like:
project/
configuration.conf
definitions.py
main.py
utils.py
In definitions.py
you can define (this requires import os
):
ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) # This is your Project Root
Thus, with the Project Root known, you can create a variable that points to the location of the configuration (this can be defined anywhere, but a logical place would be to put it in a location where constants are defined - e.g. definitions.py
):
CONFIG_PATH = os.path.join(ROOT_DIR, 'configuration.conf') # requires `import os`
Then, you can easily access the constant (in any of the other files) with the import statement (e.g. in utils.py
): from definitions import CONFIG_PATH
.
edited May 9 '18 at 17:02
answered Aug 19 '14 at 17:42
jrd1jrd1
7,12732541
7,12732541
To include the definitions.py file like that, will it be required to add a__init__.py
file to the root project directory as well ? Should that be correct ? I've just started with python and not sure on the best practices. Thanks.
– akskap
Aug 30 '16 at 8:20
1
@akskap: No, an__init__.py
will not be required, as that file is only required when defining packages: The__init__.py
files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case,__init__.py
can just be an empty file, but it can also execute initialization code for the package or set the__all__
variable, described later. See: docs.python.org/3/tutorial/modules.html#packages
– jrd1
Aug 31 '16 at 6:02
I am curious, style wise, whether it is acceptable or frowned upon to add these definitions to the__init.py__
of the root package. It would save creating another file, as well as allow the nicer syntax offrom root_pack import ROOT_DIR, CONFIG_PATH
.
– Johndt6
Jan 5 '17 at 2:30
@Johndt6: the convention is to keep__init__.py
empty, but that isn't strictly true (it's a convention after all). See this for more: stackoverflow.com/questions/2361124/using-init-py
– jrd1
Jan 5 '17 at 5:58
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:47
|
show 5 more comments
To include the definitions.py file like that, will it be required to add a__init__.py
file to the root project directory as well ? Should that be correct ? I've just started with python and not sure on the best practices. Thanks.
– akskap
Aug 30 '16 at 8:20
1
@akskap: No, an__init__.py
will not be required, as that file is only required when defining packages: The__init__.py
files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case,__init__.py
can just be an empty file, but it can also execute initialization code for the package or set the__all__
variable, described later. See: docs.python.org/3/tutorial/modules.html#packages
– jrd1
Aug 31 '16 at 6:02
I am curious, style wise, whether it is acceptable or frowned upon to add these definitions to the__init.py__
of the root package. It would save creating another file, as well as allow the nicer syntax offrom root_pack import ROOT_DIR, CONFIG_PATH
.
– Johndt6
Jan 5 '17 at 2:30
@Johndt6: the convention is to keep__init__.py
empty, but that isn't strictly true (it's a convention after all). See this for more: stackoverflow.com/questions/2361124/using-init-py
– jrd1
Jan 5 '17 at 5:58
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:47
To include the definitions.py file like that, will it be required to add a
__init__.py
file to the root project directory as well ? Should that be correct ? I've just started with python and not sure on the best practices. Thanks.– akskap
Aug 30 '16 at 8:20
To include the definitions.py file like that, will it be required to add a
__init__.py
file to the root project directory as well ? Should that be correct ? I've just started with python and not sure on the best practices. Thanks.– akskap
Aug 30 '16 at 8:20
1
1
@akskap: No, an
__init__.py
will not be required, as that file is only required when defining packages: The __init__.py
files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py
can just be an empty file, but it can also execute initialization code for the package or set the __all__
variable, described later. See: docs.python.org/3/tutorial/modules.html#packages– jrd1
Aug 31 '16 at 6:02
@akskap: No, an
__init__.py
will not be required, as that file is only required when defining packages: The __init__.py
files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py
can just be an empty file, but it can also execute initialization code for the package or set the __all__
variable, described later. See: docs.python.org/3/tutorial/modules.html#packages– jrd1
Aug 31 '16 at 6:02
I am curious, style wise, whether it is acceptable or frowned upon to add these definitions to the
__init.py__
of the root package. It would save creating another file, as well as allow the nicer syntax of from root_pack import ROOT_DIR, CONFIG_PATH
.– Johndt6
Jan 5 '17 at 2:30
I am curious, style wise, whether it is acceptable or frowned upon to add these definitions to the
__init.py__
of the root package. It would save creating another file, as well as allow the nicer syntax of from root_pack import ROOT_DIR, CONFIG_PATH
.– Johndt6
Jan 5 '17 at 2:30
@Johndt6: the convention is to keep
__init__.py
empty, but that isn't strictly true (it's a convention after all). See this for more: stackoverflow.com/questions/2361124/using-init-py– jrd1
Jan 5 '17 at 5:58
@Johndt6: the convention is to keep
__init__.py
empty, but that isn't strictly true (it's a convention after all). See this for more: stackoverflow.com/questions/2361124/using-init-py– jrd1
Jan 5 '17 at 5:58
Here
os
is not by default available. Need to import os
. So adding the line import os
would make the answer more complete.– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:47
Here
os
is not by default available. Need to import os
. So adding the line import os
would make the answer more complete.– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:47
|
show 5 more comments
To get the path of the "root" module, you can use:
import os
import sys
os.path.dirname(sys.modules['__main__'].__file__)
But more interestingly if you have an config "object" in your top-most module you could -read- from it like so:
app = sys.modules['__main__']
stuff = app.config.somefunc()
1
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:50
add a comment |
To get the path of the "root" module, you can use:
import os
import sys
os.path.dirname(sys.modules['__main__'].__file__)
But more interestingly if you have an config "object" in your top-most module you could -read- from it like so:
app = sys.modules['__main__']
stuff = app.config.somefunc()
1
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:50
add a comment |
To get the path of the "root" module, you can use:
import os
import sys
os.path.dirname(sys.modules['__main__'].__file__)
But more interestingly if you have an config "object" in your top-most module you could -read- from it like so:
app = sys.modules['__main__']
stuff = app.config.somefunc()
To get the path of the "root" module, you can use:
import os
import sys
os.path.dirname(sys.modules['__main__'].__file__)
But more interestingly if you have an config "object" in your top-most module you could -read- from it like so:
app = sys.modules['__main__']
stuff = app.config.somefunc()
edited Jul 23 '18 at 8:16
VCD
628520
628520
answered Oct 24 '16 at 20:37
DevPlayerDevPlayer
3,59011819
3,59011819
1
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:50
add a comment |
1
Hereos
is not by default available. Need to importos
. So adding the lineimport os
would make the answer more complete.
– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:50
1
1
Here
os
is not by default available. Need to import os
. So adding the line import os
would make the answer more complete.– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:50
Here
os
is not by default available. Need to import os
. So adding the line import os
would make the answer more complete.– Md. Abu Nafee Ibna Zahid
May 3 '18 at 11:50
add a comment |
Other answers advice to use file in top-level of the project. This is not necessary if you use pathlib.Path
and parent
. Consider the following directory structure where all files except README.md
and utils.py
have been omitted.
project
│ README.md
|
└───src
│ │ utils.py
| | ...
| ...
In utils.py
we define the following function.
from pathlib import Path
def get_project_root() -> Path:
"""Returns project root folder."""
return Path(__file__).parent.parent
In any module in the project we can now get the project root as follows.
from src.utils import get_project_root
root = get_project_root()
Benefits: Any module which calls get_project_root
can be moved without changing program behavior. Only when the module utils.py
is moved we have to update get_project_root
and the imports (use IDE refactoring to automate this).
add a comment |
Other answers advice to use file in top-level of the project. This is not necessary if you use pathlib.Path
and parent
. Consider the following directory structure where all files except README.md
and utils.py
have been omitted.
project
│ README.md
|
└───src
│ │ utils.py
| | ...
| ...
In utils.py
we define the following function.
from pathlib import Path
def get_project_root() -> Path:
"""Returns project root folder."""
return Path(__file__).parent.parent
In any module in the project we can now get the project root as follows.
from src.utils import get_project_root
root = get_project_root()
Benefits: Any module which calls get_project_root
can be moved without changing program behavior. Only when the module utils.py
is moved we have to update get_project_root
and the imports (use IDE refactoring to automate this).
add a comment |
Other answers advice to use file in top-level of the project. This is not necessary if you use pathlib.Path
and parent
. Consider the following directory structure where all files except README.md
and utils.py
have been omitted.
project
│ README.md
|
└───src
│ │ utils.py
| | ...
| ...
In utils.py
we define the following function.
from pathlib import Path
def get_project_root() -> Path:
"""Returns project root folder."""
return Path(__file__).parent.parent
In any module in the project we can now get the project root as follows.
from src.utils import get_project_root
root = get_project_root()
Benefits: Any module which calls get_project_root
can be moved without changing program behavior. Only when the module utils.py
is moved we have to update get_project_root
and the imports (use IDE refactoring to automate this).
Other answers advice to use file in top-level of the project. This is not necessary if you use pathlib.Path
and parent
. Consider the following directory structure where all files except README.md
and utils.py
have been omitted.
project
│ README.md
|
└───src
│ │ utils.py
| | ...
| ...
In utils.py
we define the following function.
from pathlib import Path
def get_project_root() -> Path:
"""Returns project root folder."""
return Path(__file__).parent.parent
In any module in the project we can now get the project root as follows.
from src.utils import get_project_root
root = get_project_root()
Benefits: Any module which calls get_project_root
can be moved without changing program behavior. Only when the module utils.py
is moved we have to update get_project_root
and the imports (use IDE refactoring to automate this).
edited Nov 25 '18 at 9:09
SuperShoot
1,845720
1,845720
answered Nov 25 '18 at 8:24
RikHRikH
13016
13016
add a comment |
add a comment |
A standard way to achieve this would be to use the pkg_resources
module which is part of the setuptools
package. setuptools
is used to create an install-able python package.
You can use pkg_resources
to return the contents of your desired file as a string and you can use pkg_resources
to get the actual path of the desired file on your system.
Let's say that you have a package called stackoverflow
.
stackoverflow/
|-- app
| `-- __init__.py
`-- resources
|-- bands
| |-- Dream Theater
| |-- __init__.py
| |-- King's X
| |-- Megadeth
| `-- Rush
`-- __init__.py
3 directories, 7 files
Now let's say that you want to access the file Rush from a module app.run
. Use pkg_resources.resouces_filename
to get the path to Rush and pkg_resources.resource_string
to get the contents of Rush; thusly:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('resources.bands', 'Rush')
print pkg_resources.resource_string('resources.bands', 'Rush')
The output:
/home/sri/workspace/stackoverflow/resources/bands/Rush
Base: Geddy Lee
Vocals: Geddy Lee
Guitar: Alex Lifeson
Drums: Neil Peart
This works for all packages in your python path. So if you want to know where lxml.etree
exists on your system:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('lxml', 'etree')
output:
/usr/lib64/python2.7/site-packages/lxml/etree
The point is that you can use this standard method to access files that are installed on your system (e.g pip install xxx or yum -y install python-xxx) and files that are within the module that you're currently working on.
1
I like your band choice !
– dylan_fan
Jun 20 '18 at 12:06
Ah yes. The good ol' boys. 10 bucks is 10 bucks, eh?
– shrewmouse
Jun 21 '18 at 13:42
add a comment |
A standard way to achieve this would be to use the pkg_resources
module which is part of the setuptools
package. setuptools
is used to create an install-able python package.
You can use pkg_resources
to return the contents of your desired file as a string and you can use pkg_resources
to get the actual path of the desired file on your system.
Let's say that you have a package called stackoverflow
.
stackoverflow/
|-- app
| `-- __init__.py
`-- resources
|-- bands
| |-- Dream Theater
| |-- __init__.py
| |-- King's X
| |-- Megadeth
| `-- Rush
`-- __init__.py
3 directories, 7 files
Now let's say that you want to access the file Rush from a module app.run
. Use pkg_resources.resouces_filename
to get the path to Rush and pkg_resources.resource_string
to get the contents of Rush; thusly:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('resources.bands', 'Rush')
print pkg_resources.resource_string('resources.bands', 'Rush')
The output:
/home/sri/workspace/stackoverflow/resources/bands/Rush
Base: Geddy Lee
Vocals: Geddy Lee
Guitar: Alex Lifeson
Drums: Neil Peart
This works for all packages in your python path. So if you want to know where lxml.etree
exists on your system:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('lxml', 'etree')
output:
/usr/lib64/python2.7/site-packages/lxml/etree
The point is that you can use this standard method to access files that are installed on your system (e.g pip install xxx or yum -y install python-xxx) and files that are within the module that you're currently working on.
1
I like your band choice !
– dylan_fan
Jun 20 '18 at 12:06
Ah yes. The good ol' boys. 10 bucks is 10 bucks, eh?
– shrewmouse
Jun 21 '18 at 13:42
add a comment |
A standard way to achieve this would be to use the pkg_resources
module which is part of the setuptools
package. setuptools
is used to create an install-able python package.
You can use pkg_resources
to return the contents of your desired file as a string and you can use pkg_resources
to get the actual path of the desired file on your system.
Let's say that you have a package called stackoverflow
.
stackoverflow/
|-- app
| `-- __init__.py
`-- resources
|-- bands
| |-- Dream Theater
| |-- __init__.py
| |-- King's X
| |-- Megadeth
| `-- Rush
`-- __init__.py
3 directories, 7 files
Now let's say that you want to access the file Rush from a module app.run
. Use pkg_resources.resouces_filename
to get the path to Rush and pkg_resources.resource_string
to get the contents of Rush; thusly:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('resources.bands', 'Rush')
print pkg_resources.resource_string('resources.bands', 'Rush')
The output:
/home/sri/workspace/stackoverflow/resources/bands/Rush
Base: Geddy Lee
Vocals: Geddy Lee
Guitar: Alex Lifeson
Drums: Neil Peart
This works for all packages in your python path. So if you want to know where lxml.etree
exists on your system:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('lxml', 'etree')
output:
/usr/lib64/python2.7/site-packages/lxml/etree
The point is that you can use this standard method to access files that are installed on your system (e.g pip install xxx or yum -y install python-xxx) and files that are within the module that you're currently working on.
A standard way to achieve this would be to use the pkg_resources
module which is part of the setuptools
package. setuptools
is used to create an install-able python package.
You can use pkg_resources
to return the contents of your desired file as a string and you can use pkg_resources
to get the actual path of the desired file on your system.
Let's say that you have a package called stackoverflow
.
stackoverflow/
|-- app
| `-- __init__.py
`-- resources
|-- bands
| |-- Dream Theater
| |-- __init__.py
| |-- King's X
| |-- Megadeth
| `-- Rush
`-- __init__.py
3 directories, 7 files
Now let's say that you want to access the file Rush from a module app.run
. Use pkg_resources.resouces_filename
to get the path to Rush and pkg_resources.resource_string
to get the contents of Rush; thusly:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('resources.bands', 'Rush')
print pkg_resources.resource_string('resources.bands', 'Rush')
The output:
/home/sri/workspace/stackoverflow/resources/bands/Rush
Base: Geddy Lee
Vocals: Geddy Lee
Guitar: Alex Lifeson
Drums: Neil Peart
This works for all packages in your python path. So if you want to know where lxml.etree
exists on your system:
import pkg_resources
if __name__ == "__main__":
print pkg_resources.resource_filename('lxml', 'etree')
output:
/usr/lib64/python2.7/site-packages/lxml/etree
The point is that you can use this standard method to access files that are installed on your system (e.g pip install xxx or yum -y install python-xxx) and files that are within the module that you're currently working on.
edited Aug 29 '17 at 20:38
answered Aug 29 '17 at 16:22
shrewmouseshrewmouse
1,5941319
1,5941319
1
I like your band choice !
– dylan_fan
Jun 20 '18 at 12:06
Ah yes. The good ol' boys. 10 bucks is 10 bucks, eh?
– shrewmouse
Jun 21 '18 at 13:42
add a comment |
1
I like your band choice !
– dylan_fan
Jun 20 '18 at 12:06
Ah yes. The good ol' boys. 10 bucks is 10 bucks, eh?
– shrewmouse
Jun 21 '18 at 13:42
1
1
I like your band choice !
– dylan_fan
Jun 20 '18 at 12:06
I like your band choice !
– dylan_fan
Jun 20 '18 at 12:06
Ah yes. The good ol' boys. 10 bucks is 10 bucks, eh?
– shrewmouse
Jun 21 '18 at 13:42
Ah yes. The good ol' boys. 10 bucks is 10 bucks, eh?
– shrewmouse
Jun 21 '18 at 13:42
add a comment |
All the previous solutions seem to be overly complicated for what I think you need, and often didn't work for me. The following one-line command does what you want:
import os
ROOT_DIR = os.path.abspath(os.curdir)
add a comment |
All the previous solutions seem to be overly complicated for what I think you need, and often didn't work for me. The following one-line command does what you want:
import os
ROOT_DIR = os.path.abspath(os.curdir)
add a comment |
All the previous solutions seem to be overly complicated for what I think you need, and often didn't work for me. The following one-line command does what you want:
import os
ROOT_DIR = os.path.abspath(os.curdir)
All the previous solutions seem to be overly complicated for what I think you need, and often didn't work for me. The following one-line command does what you want:
import os
ROOT_DIR = os.path.abspath(os.curdir)
answered Feb 6 at 18:27
MartimMartim
183
183
add a comment |
add a comment |
This worked for me using a standard PyCharm project with my virtual environment (venv) under the project root directory.
Code below isnt the prettiest, but consistently gets the project root. It returns the full directory path to venv from the VIRTUAL_ENV
environment variable e.g. /Users/NAME/documents/PROJECT/venv
It then splits the path at the last /
, giving an array with two elements. The first element will be the project path e.g. /Users/NAME/documents/PROJECT
import os
print(os.path.split(os.environ['VIRTUAL_ENV'])[0])
1
This won't work with setups like anaconda or pipenv, since the virtual environment isn't contained within the project in those cases.
– Gripp
Jul 6 '18 at 11:17
add a comment |
This worked for me using a standard PyCharm project with my virtual environment (venv) under the project root directory.
Code below isnt the prettiest, but consistently gets the project root. It returns the full directory path to venv from the VIRTUAL_ENV
environment variable e.g. /Users/NAME/documents/PROJECT/venv
It then splits the path at the last /
, giving an array with two elements. The first element will be the project path e.g. /Users/NAME/documents/PROJECT
import os
print(os.path.split(os.environ['VIRTUAL_ENV'])[0])
1
This won't work with setups like anaconda or pipenv, since the virtual environment isn't contained within the project in those cases.
– Gripp
Jul 6 '18 at 11:17
add a comment |
This worked for me using a standard PyCharm project with my virtual environment (venv) under the project root directory.
Code below isnt the prettiest, but consistently gets the project root. It returns the full directory path to venv from the VIRTUAL_ENV
environment variable e.g. /Users/NAME/documents/PROJECT/venv
It then splits the path at the last /
, giving an array with two elements. The first element will be the project path e.g. /Users/NAME/documents/PROJECT
import os
print(os.path.split(os.environ['VIRTUAL_ENV'])[0])
This worked for me using a standard PyCharm project with my virtual environment (venv) under the project root directory.
Code below isnt the prettiest, but consistently gets the project root. It returns the full directory path to venv from the VIRTUAL_ENV
environment variable e.g. /Users/NAME/documents/PROJECT/venv
It then splits the path at the last /
, giving an array with two elements. The first element will be the project path e.g. /Users/NAME/documents/PROJECT
import os
print(os.path.split(os.environ['VIRTUAL_ENV'])[0])
edited Apr 26 '18 at 10:54
answered Apr 26 '18 at 10:48
Gaz_EdgeGaz_Edge
8,41934280
8,41934280
1
This won't work with setups like anaconda or pipenv, since the virtual environment isn't contained within the project in those cases.
– Gripp
Jul 6 '18 at 11:17
add a comment |
1
This won't work with setups like anaconda or pipenv, since the virtual environment isn't contained within the project in those cases.
– Gripp
Jul 6 '18 at 11:17
1
1
This won't work with setups like anaconda or pipenv, since the virtual environment isn't contained within the project in those cases.
– Gripp
Jul 6 '18 at 11:17
This won't work with setups like anaconda or pipenv, since the virtual environment isn't contained within the project in those cases.
– Gripp
Jul 6 '18 at 11:17
add a comment |
I've recently been trying to do something similar and I have found these answers inadequate for my use cases (a distributed library that needs to detect project root). Mainly I've been battling different environments and platforms, and still haven't found something perfectly universal.
Code local to project
I've seen this example mentioned and used in a few places, Django, etc.
import os
print(os.path.dirname(os.path.abspath(__file__)))
Simple as this is, it only works when the file that the snippet is in is actually part of the project. We do not retrieve the project directory, but instead the snippet's directory
Similarly, the sys.modules approach breaks down when called from outside the entrypoint of the application, specifically I've observed a child thread cannot determine this without relation back to the 'main' module. I've explicitly put the import inside a function to demonstrate an import from a child thread, moving it to top level of app.py would fix it.
app/
|-- config
| `-- __init__.py
| `-- settings.py
`-- app.py
app.py
#!/usr/bin/env python
import threading
def background_setup():
# Explicitly importing this from the context of the child thread
from config import settings
print(settings.ROOT_DIR)
# Spawn a thread to background preparation tasks
t = threading.Thread(target=background_setup)
t.start()
# Do other things during initialization
t.join()
# Ready to take traffic
settings.py
import os
import sys
ROOT_DIR = None
def setup():
global ROOT_DIR
ROOT_DIR = os.path.dirname(sys.modules['__main__'].__file__)
# Do something slow
Running this program produces an attribute error:
>>> import main
>>> Exception in thread Thread-1:
Traceback (most recent call last):
File "C:Python2714libthreading.py", line 801, in __bootstrap_inner
self.run()
File "C:Python2714libthreading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "main.py", line 6, in background_setup
from config import settings
File "configsettings.py", line 34, in <module>
ROOT_DIR = get_root()
File "configsettings.py", line 31, in get_root
return os.path.dirname(sys.modules['__main__'].__file__)
AttributeError: 'module' object has no attribute '__file__'
...hence a threading-based solution
Location independent
Using the same application structure as before but modifying settings.py
import os
import sys
import inspect
import platform
import threading
ROOT_DIR = None
def setup():
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
Breaking this down:
First we want to accurately find the thread ID of the main thread. In Python3.4+ the threading library has threading.main_thread()
however, everybody doesn't use 3.4+ so we search through all threads looking for the main thread save it's ID. If the main thread has already exited, it won't be listed in the threading.enumerate()
. We raise a RuntimeError()
in this case until I find a better solution.
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
Next we find the very first stack frame of the main thread. Using the cPython specific function sys._current_frames()
we get a dictionary of every thread's current stack frame. Then utilizing inspect.getouterframes()
we can retrieve the entire stack for the main thread and the very first frame.
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
Finally, the differences between Windows and Linux implementations of inspect.getouterframes()
need to be handled. Using the cleaned up filename, os.path.abspath()
and os.path.dirname()
clean things up.
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
So far I've tested this on Python2.7 and 3.6 on Windows as well as Python3.4 on WSL
add a comment |
I've recently been trying to do something similar and I have found these answers inadequate for my use cases (a distributed library that needs to detect project root). Mainly I've been battling different environments and platforms, and still haven't found something perfectly universal.
Code local to project
I've seen this example mentioned and used in a few places, Django, etc.
import os
print(os.path.dirname(os.path.abspath(__file__)))
Simple as this is, it only works when the file that the snippet is in is actually part of the project. We do not retrieve the project directory, but instead the snippet's directory
Similarly, the sys.modules approach breaks down when called from outside the entrypoint of the application, specifically I've observed a child thread cannot determine this without relation back to the 'main' module. I've explicitly put the import inside a function to demonstrate an import from a child thread, moving it to top level of app.py would fix it.
app/
|-- config
| `-- __init__.py
| `-- settings.py
`-- app.py
app.py
#!/usr/bin/env python
import threading
def background_setup():
# Explicitly importing this from the context of the child thread
from config import settings
print(settings.ROOT_DIR)
# Spawn a thread to background preparation tasks
t = threading.Thread(target=background_setup)
t.start()
# Do other things during initialization
t.join()
# Ready to take traffic
settings.py
import os
import sys
ROOT_DIR = None
def setup():
global ROOT_DIR
ROOT_DIR = os.path.dirname(sys.modules['__main__'].__file__)
# Do something slow
Running this program produces an attribute error:
>>> import main
>>> Exception in thread Thread-1:
Traceback (most recent call last):
File "C:Python2714libthreading.py", line 801, in __bootstrap_inner
self.run()
File "C:Python2714libthreading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "main.py", line 6, in background_setup
from config import settings
File "configsettings.py", line 34, in <module>
ROOT_DIR = get_root()
File "configsettings.py", line 31, in get_root
return os.path.dirname(sys.modules['__main__'].__file__)
AttributeError: 'module' object has no attribute '__file__'
...hence a threading-based solution
Location independent
Using the same application structure as before but modifying settings.py
import os
import sys
import inspect
import platform
import threading
ROOT_DIR = None
def setup():
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
Breaking this down:
First we want to accurately find the thread ID of the main thread. In Python3.4+ the threading library has threading.main_thread()
however, everybody doesn't use 3.4+ so we search through all threads looking for the main thread save it's ID. If the main thread has already exited, it won't be listed in the threading.enumerate()
. We raise a RuntimeError()
in this case until I find a better solution.
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
Next we find the very first stack frame of the main thread. Using the cPython specific function sys._current_frames()
we get a dictionary of every thread's current stack frame. Then utilizing inspect.getouterframes()
we can retrieve the entire stack for the main thread and the very first frame.
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
Finally, the differences between Windows and Linux implementations of inspect.getouterframes()
need to be handled. Using the cleaned up filename, os.path.abspath()
and os.path.dirname()
clean things up.
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
So far I've tested this on Python2.7 and 3.6 on Windows as well as Python3.4 on WSL
add a comment |
I've recently been trying to do something similar and I have found these answers inadequate for my use cases (a distributed library that needs to detect project root). Mainly I've been battling different environments and platforms, and still haven't found something perfectly universal.
Code local to project
I've seen this example mentioned and used in a few places, Django, etc.
import os
print(os.path.dirname(os.path.abspath(__file__)))
Simple as this is, it only works when the file that the snippet is in is actually part of the project. We do not retrieve the project directory, but instead the snippet's directory
Similarly, the sys.modules approach breaks down when called from outside the entrypoint of the application, specifically I've observed a child thread cannot determine this without relation back to the 'main' module. I've explicitly put the import inside a function to demonstrate an import from a child thread, moving it to top level of app.py would fix it.
app/
|-- config
| `-- __init__.py
| `-- settings.py
`-- app.py
app.py
#!/usr/bin/env python
import threading
def background_setup():
# Explicitly importing this from the context of the child thread
from config import settings
print(settings.ROOT_DIR)
# Spawn a thread to background preparation tasks
t = threading.Thread(target=background_setup)
t.start()
# Do other things during initialization
t.join()
# Ready to take traffic
settings.py
import os
import sys
ROOT_DIR = None
def setup():
global ROOT_DIR
ROOT_DIR = os.path.dirname(sys.modules['__main__'].__file__)
# Do something slow
Running this program produces an attribute error:
>>> import main
>>> Exception in thread Thread-1:
Traceback (most recent call last):
File "C:Python2714libthreading.py", line 801, in __bootstrap_inner
self.run()
File "C:Python2714libthreading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "main.py", line 6, in background_setup
from config import settings
File "configsettings.py", line 34, in <module>
ROOT_DIR = get_root()
File "configsettings.py", line 31, in get_root
return os.path.dirname(sys.modules['__main__'].__file__)
AttributeError: 'module' object has no attribute '__file__'
...hence a threading-based solution
Location independent
Using the same application structure as before but modifying settings.py
import os
import sys
import inspect
import platform
import threading
ROOT_DIR = None
def setup():
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
Breaking this down:
First we want to accurately find the thread ID of the main thread. In Python3.4+ the threading library has threading.main_thread()
however, everybody doesn't use 3.4+ so we search through all threads looking for the main thread save it's ID. If the main thread has already exited, it won't be listed in the threading.enumerate()
. We raise a RuntimeError()
in this case until I find a better solution.
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
Next we find the very first stack frame of the main thread. Using the cPython specific function sys._current_frames()
we get a dictionary of every thread's current stack frame. Then utilizing inspect.getouterframes()
we can retrieve the entire stack for the main thread and the very first frame.
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
Finally, the differences between Windows and Linux implementations of inspect.getouterframes()
need to be handled. Using the cleaned up filename, os.path.abspath()
and os.path.dirname()
clean things up.
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
So far I've tested this on Python2.7 and 3.6 on Windows as well as Python3.4 on WSL
I've recently been trying to do something similar and I have found these answers inadequate for my use cases (a distributed library that needs to detect project root). Mainly I've been battling different environments and platforms, and still haven't found something perfectly universal.
Code local to project
I've seen this example mentioned and used in a few places, Django, etc.
import os
print(os.path.dirname(os.path.abspath(__file__)))
Simple as this is, it only works when the file that the snippet is in is actually part of the project. We do not retrieve the project directory, but instead the snippet's directory
Similarly, the sys.modules approach breaks down when called from outside the entrypoint of the application, specifically I've observed a child thread cannot determine this without relation back to the 'main' module. I've explicitly put the import inside a function to demonstrate an import from a child thread, moving it to top level of app.py would fix it.
app/
|-- config
| `-- __init__.py
| `-- settings.py
`-- app.py
app.py
#!/usr/bin/env python
import threading
def background_setup():
# Explicitly importing this from the context of the child thread
from config import settings
print(settings.ROOT_DIR)
# Spawn a thread to background preparation tasks
t = threading.Thread(target=background_setup)
t.start()
# Do other things during initialization
t.join()
# Ready to take traffic
settings.py
import os
import sys
ROOT_DIR = None
def setup():
global ROOT_DIR
ROOT_DIR = os.path.dirname(sys.modules['__main__'].__file__)
# Do something slow
Running this program produces an attribute error:
>>> import main
>>> Exception in thread Thread-1:
Traceback (most recent call last):
File "C:Python2714libthreading.py", line 801, in __bootstrap_inner
self.run()
File "C:Python2714libthreading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "main.py", line 6, in background_setup
from config import settings
File "configsettings.py", line 34, in <module>
ROOT_DIR = get_root()
File "configsettings.py", line 31, in get_root
return os.path.dirname(sys.modules['__main__'].__file__)
AttributeError: 'module' object has no attribute '__file__'
...hence a threading-based solution
Location independent
Using the same application structure as before but modifying settings.py
import os
import sys
import inspect
import platform
import threading
ROOT_DIR = None
def setup():
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
Breaking this down:
First we want to accurately find the thread ID of the main thread. In Python3.4+ the threading library has threading.main_thread()
however, everybody doesn't use 3.4+ so we search through all threads looking for the main thread save it's ID. If the main thread has already exited, it won't be listed in the threading.enumerate()
. We raise a RuntimeError()
in this case until I find a better solution.
main_id = None
for t in threading.enumerate():
if t.name == 'MainThread':
main_id = t.ident
break
if not main_id:
raise RuntimeError("Main thread exited before execution")
Next we find the very first stack frame of the main thread. Using the cPython specific function sys._current_frames()
we get a dictionary of every thread's current stack frame. Then utilizing inspect.getouterframes()
we can retrieve the entire stack for the main thread and the very first frame.
current_main_frame = sys._current_frames()[main_id]
base_frame = inspect.getouterframes(current_main_frame)[-1]
Finally, the differences between Windows and Linux implementations of inspect.getouterframes()
need to be handled. Using the cleaned up filename, os.path.abspath()
and os.path.dirname()
clean things up.
if platform.system() == 'Windows':
filename = base_frame.filename
else:
filename = base_frame[0].f_code.co_filename
global ROOT_DIR
ROOT_DIR = os.path.dirname(os.path.abspath(filename))
So far I've tested this on Python2.7 and 3.6 on Windows as well as Python3.4 on WSL
answered Sep 20 '18 at 21:18
Joseph BurnitzJoseph Burnitz
111
111
add a comment |
add a comment |
Try:
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
add a comment |
Try:
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
add a comment |
Try:
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
Try:
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
edited Jan 4 at 7:47
U9-Forward
15.8k51541
15.8k51541
answered Jan 3 at 23:54
harryharry
1417
1417
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f25389095%2fpython-get-path-of-root-project-structure%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
does
<ROOT>/__init__.py
exist?– mgilson
Aug 19 '14 at 17:08
Either your configuration file is a python module, and you can easily access it just with an import statement, either it's not a python module and you should put it in a well known location. For example $HOME/.my_project/my_project.conf.
– John Smith Optional
Aug 19 '14 at 17:09
@JohnSmithOptional - It's a JSON file. I need to be able to access it using the path. Yes. All of the folders include it.
– Shookie
Aug 19 '14 at 17:09
_ It's okay to assume that we know the project root's name._ Does that mean you know the path to the project? Isn't it just os.path.join(known_root_name, "configuration.conf") then?
– tdelaney
Aug 19 '14 at 17:19
If it's a user configuration I'd generally use something like
os.path.expanduser('~/.myproject/myproject.conf')
. It works on Unix and Windows.– John Smith Optional
Aug 19 '14 at 17:22