How can I override a setuptools command from a setup_require dependancy?












1















Context:



In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py that compiles them (through a copy of pymsgfmt) before calling its base class:



from setuptools.command.build_py import build_py as _build_py

class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)

def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs


Then I have only to declare it in the cmdclass parameter of setup:



setup(
...
cmdclass = {"build_py", mypgk.build_py},
)


So far, so good, when my module is installed and imported in the setup.py script, the build phase of setuptools correctly process my po files.



Problem:



The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires, or setup_requires parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py is runned, but it cannot run without mypkg being first installed.



Current research:



I have tried to use the magic entry_points to declare the build_py override in mypkg setup.py script:



...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}


but it has no effects, while I can declare a working new build_mo command that way:



entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}


Long story short, python setup.py build_mo calls my override, while python setup.py build_py calls setuptools version.



Question:



Why does my attempt of overriding the build_py command with an entry_points declaration not work, and how could I do it?










share|improve this question

























  • I use the build_py step and not the build_ext because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.

    – Serge Ballesta
    Nov 26 '18 at 10:04











  • From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?

    – hoefling
    Nov 26 '18 at 20:13











  • @hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.

    – Serge Ballesta
    Nov 26 '18 at 22:42
















1















Context:



In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py that compiles them (through a copy of pymsgfmt) before calling its base class:



from setuptools.command.build_py import build_py as _build_py

class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)

def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs


Then I have only to declare it in the cmdclass parameter of setup:



setup(
...
cmdclass = {"build_py", mypgk.build_py},
)


So far, so good, when my module is installed and imported in the setup.py script, the build phase of setuptools correctly process my po files.



Problem:



The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires, or setup_requires parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py is runned, but it cannot run without mypkg being first installed.



Current research:



I have tried to use the magic entry_points to declare the build_py override in mypkg setup.py script:



...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}


but it has no effects, while I can declare a working new build_mo command that way:



entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}


Long story short, python setup.py build_mo calls my override, while python setup.py build_py calls setuptools version.



Question:



Why does my attempt of overriding the build_py command with an entry_points declaration not work, and how could I do it?










share|improve this question

























  • I use the build_py step and not the build_ext because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.

    – Serge Ballesta
    Nov 26 '18 at 10:04











  • From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?

    – hoefling
    Nov 26 '18 at 20:13











  • @hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.

    – Serge Ballesta
    Nov 26 '18 at 22:42














1












1








1








Context:



In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py that compiles them (through a copy of pymsgfmt) before calling its base class:



from setuptools.command.build_py import build_py as _build_py

class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)

def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs


Then I have only to declare it in the cmdclass parameter of setup:



setup(
...
cmdclass = {"build_py", mypgk.build_py},
)


So far, so good, when my module is installed and imported in the setup.py script, the build phase of setuptools correctly process my po files.



Problem:



The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires, or setup_requires parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py is runned, but it cannot run without mypkg being first installed.



Current research:



I have tried to use the magic entry_points to declare the build_py override in mypkg setup.py script:



...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}


but it has no effects, while I can declare a working new build_mo command that way:



entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}


Long story short, python setup.py build_mo calls my override, while python setup.py build_py calls setuptools version.



Question:



Why does my attempt of overriding the build_py command with an entry_points declaration not work, and how could I do it?










share|improve this question
















Context:



In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py that compiles them (through a copy of pymsgfmt) before calling its base class:



from setuptools.command.build_py import build_py as _build_py

class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)

def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs


Then I have only to declare it in the cmdclass parameter of setup:



setup(
...
cmdclass = {"build_py", mypgk.build_py},
)


So far, so good, when my module is installed and imported in the setup.py script, the build phase of setuptools correctly process my po files.



Problem:



The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires, or setup_requires parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py is runned, but it cannot run without mypkg being first installed.



Current research:



I have tried to use the magic entry_points to declare the build_py override in mypkg setup.py script:



...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}


but it has no effects, while I can declare a working new build_mo command that way:



entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}


Long story short, python setup.py build_mo calls my override, while python setup.py build_py calls setuptools version.



Question:



Why does my attempt of overriding the build_py command with an entry_points declaration not work, and how could I do it?







python override setuptools






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 26 '18 at 22:40







Serge Ballesta

















asked Nov 26 '18 at 9:49









Serge BallestaSerge Ballesta

78.1k957130




78.1k957130













  • I use the build_py step and not the build_ext because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.

    – Serge Ballesta
    Nov 26 '18 at 10:04











  • From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?

    – hoefling
    Nov 26 '18 at 20:13











  • @hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.

    – Serge Ballesta
    Nov 26 '18 at 22:42



















  • I use the build_py step and not the build_ext because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.

    – Serge Ballesta
    Nov 26 '18 at 10:04











  • From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?

    – hoefling
    Nov 26 '18 at 20:13











  • @hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.

    – Serge Ballesta
    Nov 26 '18 at 22:42

















I use the build_py step and not the build_ext because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.

– Serge Ballesta
Nov 26 '18 at 10:04





I use the build_py step and not the build_ext because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.

– Serge Ballesta
Nov 26 '18 at 10:04













From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?

– hoefling
Nov 26 '18 at 20:13





From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?

– hoefling
Nov 26 '18 at 20:13













@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.

– Serge Ballesta
Nov 26 '18 at 22:42





@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.

– Serge Ballesta
Nov 26 '18 at 22:42












1 Answer
1






active

oldest

votes


















0














Explaining the problem



I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.



That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.



I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.



Possible solution:



As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:



from distutils.command.build import build as _build

class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)


In my test, it is enough to make setuptools use the custom build_py class with a simple



setup(
...
setup_requires = ["mypkg"],
)





share|improve this answer

























    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53478445%2fhow-can-i-override-a-setuptools-command-from-a-setup-require-dependancy%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    Explaining the problem



    I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.



    That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.



    I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.



    Possible solution:



    As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:



    from distutils.command.build import build as _build

    class build(_build):
    parent = _build
    def run(self):
    self.distribution.cmdclass["build_py"] = build_py
    self.parent.run(self)


    In my test, it is enough to make setuptools use the custom build_py class with a simple



    setup(
    ...
    setup_requires = ["mypkg"],
    )





    share|improve this answer






























      0














      Explaining the problem



      I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.



      That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.



      I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.



      Possible solution:



      As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:



      from distutils.command.build import build as _build

      class build(_build):
      parent = _build
      def run(self):
      self.distribution.cmdclass["build_py"] = build_py
      self.parent.run(self)


      In my test, it is enough to make setuptools use the custom build_py class with a simple



      setup(
      ...
      setup_requires = ["mypkg"],
      )





      share|improve this answer




























        0












        0








        0







        Explaining the problem



        I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.



        That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.



        I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.



        Possible solution:



        As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:



        from distutils.command.build import build as _build

        class build(_build):
        parent = _build
        def run(self):
        self.distribution.cmdclass["build_py"] = build_py
        self.parent.run(self)


        In my test, it is enough to make setuptools use the custom build_py class with a simple



        setup(
        ...
        setup_requires = ["mypkg"],
        )





        share|improve this answer















        Explaining the problem



        I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.



        That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.



        I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.



        Possible solution:



        As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:



        from distutils.command.build import build as _build

        class build(_build):
        parent = _build
        def run(self):
        self.distribution.cmdclass["build_py"] = build_py
        self.parent.run(self)


        In my test, it is enough to make setuptools use the custom build_py class with a simple



        setup(
        ...
        setup_requires = ["mypkg"],
        )






        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Nov 26 '18 at 22:45

























        answered Nov 26 '18 at 14:00









        Serge BallestaSerge Ballesta

        78.1k957130




        78.1k957130
































            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53478445%2fhow-can-i-override-a-setuptools-command-from-a-setup-require-dependancy%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

            Calculate evaluation metrics using cross_val_predict sklearn

            Insert data from modal to MySQL (multiple modal on website)