Are Django settings shared across uwsgi workers?











up vote
4
down vote

favorite












I have a Django app, with a setting (in my settings.py file) that's populated dynamically in my App Config's ready() function. Ie in settings.py I have:



POPULATE_THIS = None


and then in apps.py in ready I have:



def ready(self):
if POPULATE_THIS is None:
POPULATE_THIS = ... some code which instantiates an object I need that's effectively a singleton ...


This seems to work ok. But I'm now at the point where rather than just running the dev server locally (ie python manage.py runserver), I'm now running the Django app through uwsgi (proxied behind nginx), and uwsgi is configured to run 10 worker processes (ie my uwsgi ini file has processes = 10 and threads = 1).



I'm seeing evidence that even though there are 10 uwsgi processes , ready() is still called exactly once on app startup and the value of POPULATE_THIS is the same across all workers (doing a str on it is giving the same memory address).



My question: How is that value shared across the uwsgi processes, as I thought separate processes are distinct and do not share any memory? And am I correct in assuming that ready() is going to be called once per app startup (ie when uwsgi itself spins up), and not once per uwsgi worker process startup?



This answer (Multiple server processes using nginx and uWSGI) on a different question seems to indicate that some data is shared across workers, but I can't seem to find any official docs that indicate what exactly is shared and how, specifically with respect to Django settings, so some explanation/details would be much appreciated.










share|improve this question


















  • 1




    Not quite sure about the answer, just some thoughts: first, the fact that the var POPULATE_THIS points to the same memory address in every uwsgi process does not mean that they share memory since this address is virtual (not physical) and every process has the full (with exceptions) virtual memory adresses space available. One possible explanation for both of your observations would be that uwsgi forks the processes AFTER every app has been initialized. I don't know if that is the case but it could be...
    – ivissani
    Nov 19 at 23:38






  • 1




    As far as I know, that value can not be shared across processses, but your tests suggest otherwise. Maybe as @ivissani suggested, uwsgi may be forking processes after initialization. You can write a view method that assigns a random value to POPULATE_THIS and another one to return it, and call these methods several times through requests to see how they behave.
    – Ozgur Akcali
    Nov 20 at 10:58












  • ^^ Great tip, I did this and what I'm seeing is it looks like it does fork the process, so each worker gets an independent copy of the settings. If I update a settings value in one of the workers, that change isn't visible in the other(s).
    – Adam Parkin
    Nov 20 at 16:15















up vote
4
down vote

favorite












I have a Django app, with a setting (in my settings.py file) that's populated dynamically in my App Config's ready() function. Ie in settings.py I have:



POPULATE_THIS = None


and then in apps.py in ready I have:



def ready(self):
if POPULATE_THIS is None:
POPULATE_THIS = ... some code which instantiates an object I need that's effectively a singleton ...


This seems to work ok. But I'm now at the point where rather than just running the dev server locally (ie python manage.py runserver), I'm now running the Django app through uwsgi (proxied behind nginx), and uwsgi is configured to run 10 worker processes (ie my uwsgi ini file has processes = 10 and threads = 1).



I'm seeing evidence that even though there are 10 uwsgi processes , ready() is still called exactly once on app startup and the value of POPULATE_THIS is the same across all workers (doing a str on it is giving the same memory address).



My question: How is that value shared across the uwsgi processes, as I thought separate processes are distinct and do not share any memory? And am I correct in assuming that ready() is going to be called once per app startup (ie when uwsgi itself spins up), and not once per uwsgi worker process startup?



This answer (Multiple server processes using nginx and uWSGI) on a different question seems to indicate that some data is shared across workers, but I can't seem to find any official docs that indicate what exactly is shared and how, specifically with respect to Django settings, so some explanation/details would be much appreciated.










share|improve this question


















  • 1




    Not quite sure about the answer, just some thoughts: first, the fact that the var POPULATE_THIS points to the same memory address in every uwsgi process does not mean that they share memory since this address is virtual (not physical) and every process has the full (with exceptions) virtual memory adresses space available. One possible explanation for both of your observations would be that uwsgi forks the processes AFTER every app has been initialized. I don't know if that is the case but it could be...
    – ivissani
    Nov 19 at 23:38






  • 1




    As far as I know, that value can not be shared across processses, but your tests suggest otherwise. Maybe as @ivissani suggested, uwsgi may be forking processes after initialization. You can write a view method that assigns a random value to POPULATE_THIS and another one to return it, and call these methods several times through requests to see how they behave.
    – Ozgur Akcali
    Nov 20 at 10:58












  • ^^ Great tip, I did this and what I'm seeing is it looks like it does fork the process, so each worker gets an independent copy of the settings. If I update a settings value in one of the workers, that change isn't visible in the other(s).
    – Adam Parkin
    Nov 20 at 16:15













up vote
4
down vote

favorite









up vote
4
down vote

favorite











I have a Django app, with a setting (in my settings.py file) that's populated dynamically in my App Config's ready() function. Ie in settings.py I have:



POPULATE_THIS = None


and then in apps.py in ready I have:



def ready(self):
if POPULATE_THIS is None:
POPULATE_THIS = ... some code which instantiates an object I need that's effectively a singleton ...


This seems to work ok. But I'm now at the point where rather than just running the dev server locally (ie python manage.py runserver), I'm now running the Django app through uwsgi (proxied behind nginx), and uwsgi is configured to run 10 worker processes (ie my uwsgi ini file has processes = 10 and threads = 1).



I'm seeing evidence that even though there are 10 uwsgi processes , ready() is still called exactly once on app startup and the value of POPULATE_THIS is the same across all workers (doing a str on it is giving the same memory address).



My question: How is that value shared across the uwsgi processes, as I thought separate processes are distinct and do not share any memory? And am I correct in assuming that ready() is going to be called once per app startup (ie when uwsgi itself spins up), and not once per uwsgi worker process startup?



This answer (Multiple server processes using nginx and uWSGI) on a different question seems to indicate that some data is shared across workers, but I can't seem to find any official docs that indicate what exactly is shared and how, specifically with respect to Django settings, so some explanation/details would be much appreciated.










share|improve this question













I have a Django app, with a setting (in my settings.py file) that's populated dynamically in my App Config's ready() function. Ie in settings.py I have:



POPULATE_THIS = None


and then in apps.py in ready I have:



def ready(self):
if POPULATE_THIS is None:
POPULATE_THIS = ... some code which instantiates an object I need that's effectively a singleton ...


This seems to work ok. But I'm now at the point where rather than just running the dev server locally (ie python manage.py runserver), I'm now running the Django app through uwsgi (proxied behind nginx), and uwsgi is configured to run 10 worker processes (ie my uwsgi ini file has processes = 10 and threads = 1).



I'm seeing evidence that even though there are 10 uwsgi processes , ready() is still called exactly once on app startup and the value of POPULATE_THIS is the same across all workers (doing a str on it is giving the same memory address).



My question: How is that value shared across the uwsgi processes, as I thought separate processes are distinct and do not share any memory? And am I correct in assuming that ready() is going to be called once per app startup (ie when uwsgi itself spins up), and not once per uwsgi worker process startup?



This answer (Multiple server processes using nginx and uWSGI) on a different question seems to indicate that some data is shared across workers, but I can't seem to find any official docs that indicate what exactly is shared and how, specifically with respect to Django settings, so some explanation/details would be much appreciated.







django python-3.x uwsgi






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 19 at 23:23









Adam Parkin

7,28794569




7,28794569








  • 1




    Not quite sure about the answer, just some thoughts: first, the fact that the var POPULATE_THIS points to the same memory address in every uwsgi process does not mean that they share memory since this address is virtual (not physical) and every process has the full (with exceptions) virtual memory adresses space available. One possible explanation for both of your observations would be that uwsgi forks the processes AFTER every app has been initialized. I don't know if that is the case but it could be...
    – ivissani
    Nov 19 at 23:38






  • 1




    As far as I know, that value can not be shared across processses, but your tests suggest otherwise. Maybe as @ivissani suggested, uwsgi may be forking processes after initialization. You can write a view method that assigns a random value to POPULATE_THIS and another one to return it, and call these methods several times through requests to see how they behave.
    – Ozgur Akcali
    Nov 20 at 10:58












  • ^^ Great tip, I did this and what I'm seeing is it looks like it does fork the process, so each worker gets an independent copy of the settings. If I update a settings value in one of the workers, that change isn't visible in the other(s).
    – Adam Parkin
    Nov 20 at 16:15














  • 1




    Not quite sure about the answer, just some thoughts: first, the fact that the var POPULATE_THIS points to the same memory address in every uwsgi process does not mean that they share memory since this address is virtual (not physical) and every process has the full (with exceptions) virtual memory adresses space available. One possible explanation for both of your observations would be that uwsgi forks the processes AFTER every app has been initialized. I don't know if that is the case but it could be...
    – ivissani
    Nov 19 at 23:38






  • 1




    As far as I know, that value can not be shared across processses, but your tests suggest otherwise. Maybe as @ivissani suggested, uwsgi may be forking processes after initialization. You can write a view method that assigns a random value to POPULATE_THIS and another one to return it, and call these methods several times through requests to see how they behave.
    – Ozgur Akcali
    Nov 20 at 10:58












  • ^^ Great tip, I did this and what I'm seeing is it looks like it does fork the process, so each worker gets an independent copy of the settings. If I update a settings value in one of the workers, that change isn't visible in the other(s).
    – Adam Parkin
    Nov 20 at 16:15








1




1




Not quite sure about the answer, just some thoughts: first, the fact that the var POPULATE_THIS points to the same memory address in every uwsgi process does not mean that they share memory since this address is virtual (not physical) and every process has the full (with exceptions) virtual memory adresses space available. One possible explanation for both of your observations would be that uwsgi forks the processes AFTER every app has been initialized. I don't know if that is the case but it could be...
– ivissani
Nov 19 at 23:38




Not quite sure about the answer, just some thoughts: first, the fact that the var POPULATE_THIS points to the same memory address in every uwsgi process does not mean that they share memory since this address is virtual (not physical) and every process has the full (with exceptions) virtual memory adresses space available. One possible explanation for both of your observations would be that uwsgi forks the processes AFTER every app has been initialized. I don't know if that is the case but it could be...
– ivissani
Nov 19 at 23:38




1




1




As far as I know, that value can not be shared across processses, but your tests suggest otherwise. Maybe as @ivissani suggested, uwsgi may be forking processes after initialization. You can write a view method that assigns a random value to POPULATE_THIS and another one to return it, and call these methods several times through requests to see how they behave.
– Ozgur Akcali
Nov 20 at 10:58






As far as I know, that value can not be shared across processses, but your tests suggest otherwise. Maybe as @ivissani suggested, uwsgi may be forking processes after initialization. You can write a view method that assigns a random value to POPULATE_THIS and another one to return it, and call these methods several times through requests to see how they behave.
– Ozgur Akcali
Nov 20 at 10:58














^^ Great tip, I did this and what I'm seeing is it looks like it does fork the process, so each worker gets an independent copy of the settings. If I update a settings value in one of the workers, that change isn't visible in the other(s).
– Adam Parkin
Nov 20 at 16:15




^^ Great tip, I did this and what I'm seeing is it looks like it does fork the process, so each worker gets an independent copy of the settings. If I update a settings value in one of the workers, that change isn't visible in the other(s).
– Adam Parkin
Nov 20 at 16:15












1 Answer
1






active

oldest

votes

















up vote
0
down vote













Exactly.



It seems that uwsgi only spam processes of the django application itself, therefore all the functions ready will be called only once, during the first run.






share|improve this answer





















    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53384081%2fare-django-settings-shared-across-uwsgi-workers%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    0
    down vote













    Exactly.



    It seems that uwsgi only spam processes of the django application itself, therefore all the functions ready will be called only once, during the first run.






    share|improve this answer

























      up vote
      0
      down vote













      Exactly.



      It seems that uwsgi only spam processes of the django application itself, therefore all the functions ready will be called only once, during the first run.






      share|improve this answer























        up vote
        0
        down vote










        up vote
        0
        down vote









        Exactly.



        It seems that uwsgi only spam processes of the django application itself, therefore all the functions ready will be called only once, during the first run.






        share|improve this answer












        Exactly.



        It seems that uwsgi only spam processes of the django application itself, therefore all the functions ready will be called only once, during the first run.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 19 at 23:34









        Marcelo Duarte Fernandes

        111




        111






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53384081%2fare-django-settings-shared-across-uwsgi-workers%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Feedback on college project

            Futebolista

            Albești (Vaslui)