How can I override a setuptools command from a setup_require dependancy?
Context:
In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py that compiles them (through a copy of pymsgfmt) before calling its base class:
from setuptools.command.build_py import build_py as _build_py
class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)
def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs
Then I have only to declare it in the cmdclass parameter of setup:
setup(
...
cmdclass = {"build_py", mypgk.build_py},
)
So far, so good, when my module is installed and imported in the setup.py script, the build phase of setuptools correctly process my po files.
Problem:
The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires, or setup_requires parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py is runned, but it cannot run without mypkg being first installed.
Current research:
I have tried to use the magic entry_points to declare the build_py override in mypkg setup.py script:
...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}
but it has no effects, while I can declare a working new build_mo command that way:
entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}
Long story short, python setup.py build_mo calls my override, while python setup.py build_py calls setuptools version.
Question:
Why does my attempt of overriding the build_py command with an entry_points declaration not work, and how could I do it?
python override setuptools
add a comment |
Context:
In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py that compiles them (through a copy of pymsgfmt) before calling its base class:
from setuptools.command.build_py import build_py as _build_py
class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)
def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs
Then I have only to declare it in the cmdclass parameter of setup:
setup(
...
cmdclass = {"build_py", mypgk.build_py},
)
So far, so good, when my module is installed and imported in the setup.py script, the build phase of setuptools correctly process my po files.
Problem:
The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires, or setup_requires parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py is runned, but it cannot run without mypkg being first installed.
Current research:
I have tried to use the magic entry_points to declare the build_py override in mypkg setup.py script:
...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}
but it has no effects, while I can declare a working new build_mo command that way:
entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}
Long story short, python setup.py build_mo calls my override, while python setup.py build_py calls setuptools version.
Question:
Why does my attempt of overriding the build_py command with an entry_points declaration not work, and how could I do it?
python override setuptools
I use thebuild_pystep and not thebuild_extbecause I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.
– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
add a comment |
Context:
In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py that compiles them (through a copy of pymsgfmt) before calling its base class:
from setuptools.command.build_py import build_py as _build_py
class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)
def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs
Then I have only to declare it in the cmdclass parameter of setup:
setup(
...
cmdclass = {"build_py", mypgk.build_py},
)
So far, so good, when my module is installed and imported in the setup.py script, the build phase of setuptools correctly process my po files.
Problem:
The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires, or setup_requires parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py is runned, but it cannot run without mypkg being first installed.
Current research:
I have tried to use the magic entry_points to declare the build_py override in mypkg setup.py script:
...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}
but it has no effects, while I can declare a working new build_mo command that way:
entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}
Long story short, python setup.py build_mo calls my override, while python setup.py build_py calls setuptools version.
Question:
Why does my attempt of overriding the build_py command with an entry_points declaration not work, and how could I do it?
python override setuptools
Context:
In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py that compiles them (through a copy of pymsgfmt) before calling its base class:
from setuptools.command.build_py import build_py as _build_py
class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)
def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs
Then I have only to declare it in the cmdclass parameter of setup:
setup(
...
cmdclass = {"build_py", mypgk.build_py},
)
So far, so good, when my module is installed and imported in the setup.py script, the build phase of setuptools correctly process my po files.
Problem:
The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires, or setup_requires parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py is runned, but it cannot run without mypkg being first installed.
Current research:
I have tried to use the magic entry_points to declare the build_py override in mypkg setup.py script:
...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}
but it has no effects, while I can declare a working new build_mo command that way:
entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}
Long story short, python setup.py build_mo calls my override, while python setup.py build_py calls setuptools version.
Question:
Why does my attempt of overriding the build_py command with an entry_points declaration not work, and how could I do it?
python override setuptools
python override setuptools
edited Nov 26 '18 at 22:40
Serge Ballesta
asked Nov 26 '18 at 9:49
Serge BallestaSerge Ballesta
78.1k957130
78.1k957130
I use thebuild_pystep and not thebuild_extbecause I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.
– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
add a comment |
I use thebuild_pystep and not thebuild_extbecause I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.
– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
I use the
build_py step and not the build_ext because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.– Serge Ballesta
Nov 26 '18 at 10:04
I use the
build_py step and not the build_ext because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
add a comment |
1 Answer
1
active
oldest
votes
Explaining the problem
I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.
That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.
I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.
Possible solution:
As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools use the custom build_py class with a simple
setup(
...
setup_requires = ["mypkg"],
)
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53478445%2fhow-can-i-override-a-setuptools-command-from-a-setup-require-dependancy%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Explaining the problem
I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.
That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.
I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.
Possible solution:
As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools use the custom build_py class with a simple
setup(
...
setup_requires = ["mypkg"],
)
add a comment |
Explaining the problem
I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.
That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.
I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.
Possible solution:
As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools use the custom build_py class with a simple
setup(
...
setup_requires = ["mypkg"],
)
add a comment |
Explaining the problem
I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.
That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.
I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.
Possible solution:
As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools use the custom build_py class with a simple
setup(
...
setup_requires = ["mypkg"],
)
Explaining the problem
I was close to the solution. After some more research in setuptools docs and sources, I finally realized that it already used the entry_points machinery to override distutils commands by its own ones.
That means that when you try to override a setuptools command, in fact you propose a second override for the same command. Because of the way setuptools processes it, only the first override found is used, and from my tests, setuptools one is that first.
I can now say that because of that, only commands from distutils that are not overriden in setuptools can be processed that way. The good news is that build is not overriden and that in normal use, build_py is always called from build.
Possible solution:
As the build command is not overriden by setuptools, it is easy to replace it with an entry_point. Then the custom build command class can update the cmdclass directory to declare the custom build_py class because the base build loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools use the custom build_py class with a simple
setup(
...
setup_requires = ["mypkg"],
)
edited Nov 26 '18 at 22:45
answered Nov 26 '18 at 14:00
Serge BallestaSerge Ballesta
78.1k957130
78.1k957130
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53478445%2fhow-can-i-override-a-setuptools-command-from-a-setup-require-dependancy%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I use the
build_pystep and not thebuild_extbecause I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42