How can I override a setuptools command from a setup_require dependancy?
Context:
In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py
that compiles them (through a copy of pymsgfmt) before calling its base class:
from setuptools.command.build_py import build_py as _build_py
class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)
def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs
Then I have only to declare it in the cmdclass
parameter of setup
:
setup(
...
cmdclass = {"build_py", mypgk.build_py},
)
So far, so good, when my module is installed and imported in the setup.py
script, the build phase of setuptools correctly process my po files.
Problem:
The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires
, or setup_requires
parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py
is runned, but it cannot run without mypkg
being first installed.
Current research:
I have tried to use the magic entry_points
to declare the build_py
override in mypkg
setup.py
script:
...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}
but it has no effects, while I can declare a working new build_mo
command that way:
entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}
Long story short, python setup.py build_mo
calls my override, while python setup.py build_py
calls setuptools version.
Question:
Why does my attempt of overriding the build_py
command with an entry_points
declaration not work, and how could I do it?
python override setuptools
add a comment |
Context:
In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py
that compiles them (through a copy of pymsgfmt) before calling its base class:
from setuptools.command.build_py import build_py as _build_py
class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)
def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs
Then I have only to declare it in the cmdclass
parameter of setup
:
setup(
...
cmdclass = {"build_py", mypgk.build_py},
)
So far, so good, when my module is installed and imported in the setup.py
script, the build phase of setuptools correctly process my po files.
Problem:
The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires
, or setup_requires
parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py
is runned, but it cannot run without mypkg
being first installed.
Current research:
I have tried to use the magic entry_points
to declare the build_py
override in mypkg
setup.py
script:
...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}
but it has no effects, while I can declare a working new build_mo
command that way:
entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}
Long story short, python setup.py build_mo
calls my override, while python setup.py build_py
calls setuptools version.
Question:
Why does my attempt of overriding the build_py
command with an entry_points
declaration not work, and how could I do it?
python override setuptools
I use thebuild_py
step and not thebuild_ext
because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.
– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
add a comment |
Context:
In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py
that compiles them (through a copy of pymsgfmt) before calling its base class:
from setuptools.command.build_py import build_py as _build_py
class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)
def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs
Then I have only to declare it in the cmdclass
parameter of setup
:
setup(
...
cmdclass = {"build_py", mypgk.build_py},
)
So far, so good, when my module is installed and imported in the setup.py
script, the build phase of setuptools correctly process my po files.
Problem:
The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires
, or setup_requires
parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py
is runned, but it cannot run without mypkg
being first installed.
Current research:
I have tried to use the magic entry_points
to declare the build_py
override in mypkg
setup.py
script:
...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}
but it has no effects, while I can declare a working new build_mo
command that way:
entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}
Long story short, python setup.py build_mo
calls my override, while python setup.py build_py
calls setuptools version.
Question:
Why does my attempt of overriding the build_py
command with an entry_points
declaration not work, and how could I do it?
python override setuptools
Context:
In order to be able to translate GNU gettext po files into mo ones at build time in setuptools, I have created a subclass of setuptools.command.build_py
that compiles them (through a copy of pymsgfmt) before calling its base class:
from setuptools.command.build_py import build_py as _build_py
class build_py(_build_py):
parent = _build_py
def run(self):
self.compile_po_files() # internal implementation
self.parent.run(self)
def get_outputs(self): # overriden to produce a correct list of installed files
build_mo = self.get_finalized_command("build_mo")
return _build_py.get_outputs(self) + self.outputs
Then I have only to declare it in the cmdclass
parameter of setup
:
setup(
...
cmdclass = {"build_py", mypgk.build_py},
)
So far, so good, when my module is installed and imported in the setup.py
script, the build phase of setuptools correctly process my po files.
Problem:
The goal would be to allow a simple installation of a source distribution with pip. Things look nice, because pip takes care of any dependency provided they are declared in a install_requires
, or setup_requires
parameter. And this is where the chicken and egg problem lies: the dependancy are installed when setup.py
is runned, but it cannot run without mypkg
being first installed.
Current research:
I have tried to use the magic entry_points
to declare the build_py
override in mypkg
setup.py
script:
...
entry_points = {
"distutils.commands": [
"build_py = mypkg:build_py",
],
}
but it has no effects, while I can declare a working new build_mo
command that way:
entry_points = {
"distutils.commands": [
"build_mo = mypkg:build_py",
],
}
Long story short, python setup.py build_mo
calls my override, while python setup.py build_py
calls setuptools version.
Question:
Why does my attempt of overriding the build_py
command with an entry_points
declaration not work, and how could I do it?
python override setuptools
python override setuptools
edited Nov 26 '18 at 22:40
Serge Ballesta
asked Nov 26 '18 at 9:49
Serge BallestaSerge Ballesta
78.1k957130
78.1k957130
I use thebuild_py
step and not thebuild_ext
because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.
– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
add a comment |
I use thebuild_py
step and not thebuild_ext
because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.
– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
I use the
build_py
step and not the build_ext
because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.– Serge Ballesta
Nov 26 '18 at 10:04
I use the
build_py
step and not the build_ext
because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42
add a comment |
1 Answer
1
active
oldest
votes
Explaining the problem
I was close to the solution. After some more research in setuptools
docs and sources, I finally realized that it already used the entry_points
machinery to override distutils
commands by its own ones.
That means that when you try to override a setuptools
command, in fact you propose a second override for the same command. Because of the way setuptools
processes it, only the first override found is used, and from my tests, setuptools
one is that first.
I can now say that because of that, only commands from distutils
that are not overriden in setuptools
can be processed that way. The good news is that build
is not overriden and that in normal use, build_py
is always called from build
.
Possible solution:
As the build
command is not overriden by setuptools
, it is easy to replace it with an entry_point
. Then the custom build
command class can update the cmdclass
directory to declare the custom build_py
class because the base build
loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools
use the custom build_py
class with a simple
setup(
...
setup_requires = ["mypkg"],
)
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53478445%2fhow-can-i-override-a-setuptools-command-from-a-setup-require-dependancy%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Explaining the problem
I was close to the solution. After some more research in setuptools
docs and sources, I finally realized that it already used the entry_points
machinery to override distutils
commands by its own ones.
That means that when you try to override a setuptools
command, in fact you propose a second override for the same command. Because of the way setuptools
processes it, only the first override found is used, and from my tests, setuptools
one is that first.
I can now say that because of that, only commands from distutils
that are not overriden in setuptools
can be processed that way. The good news is that build
is not overriden and that in normal use, build_py
is always called from build
.
Possible solution:
As the build
command is not overriden by setuptools
, it is easy to replace it with an entry_point
. Then the custom build
command class can update the cmdclass
directory to declare the custom build_py
class because the base build
loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools
use the custom build_py
class with a simple
setup(
...
setup_requires = ["mypkg"],
)
add a comment |
Explaining the problem
I was close to the solution. After some more research in setuptools
docs and sources, I finally realized that it already used the entry_points
machinery to override distutils
commands by its own ones.
That means that when you try to override a setuptools
command, in fact you propose a second override for the same command. Because of the way setuptools
processes it, only the first override found is used, and from my tests, setuptools
one is that first.
I can now say that because of that, only commands from distutils
that are not overriden in setuptools
can be processed that way. The good news is that build
is not overriden and that in normal use, build_py
is always called from build
.
Possible solution:
As the build
command is not overriden by setuptools
, it is easy to replace it with an entry_point
. Then the custom build
command class can update the cmdclass
directory to declare the custom build_py
class because the base build
loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools
use the custom build_py
class with a simple
setup(
...
setup_requires = ["mypkg"],
)
add a comment |
Explaining the problem
I was close to the solution. After some more research in setuptools
docs and sources, I finally realized that it already used the entry_points
machinery to override distutils
commands by its own ones.
That means that when you try to override a setuptools
command, in fact you propose a second override for the same command. Because of the way setuptools
processes it, only the first override found is used, and from my tests, setuptools
one is that first.
I can now say that because of that, only commands from distutils
that are not overriden in setuptools
can be processed that way. The good news is that build
is not overriden and that in normal use, build_py
is always called from build
.
Possible solution:
As the build
command is not overriden by setuptools
, it is easy to replace it with an entry_point
. Then the custom build
command class can update the cmdclass
directory to declare the custom build_py
class because the base build
loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools
use the custom build_py
class with a simple
setup(
...
setup_requires = ["mypkg"],
)
Explaining the problem
I was close to the solution. After some more research in setuptools
docs and sources, I finally realized that it already used the entry_points
machinery to override distutils
commands by its own ones.
That means that when you try to override a setuptools
command, in fact you propose a second override for the same command. Because of the way setuptools
processes it, only the first override found is used, and from my tests, setuptools
one is that first.
I can now say that because of that, only commands from distutils
that are not overriden in setuptools
can be processed that way. The good news is that build
is not overriden and that in normal use, build_py
is always called from build
.
Possible solution:
As the build
command is not overriden by setuptools
, it is easy to replace it with an entry_point
. Then the custom build
command class can update the cmdclass
directory to declare the custom build_py
class because the base build
loads it. The code can be:
from distutils.command.build import build as _build
class build(_build):
parent = _build
def run(self):
self.distribution.cmdclass["build_py"] = build_py
self.parent.run(self)
In my test, it is enough to make setuptools
use the custom build_py
class with a simple
setup(
...
setup_requires = ["mypkg"],
)
edited Nov 26 '18 at 22:45
answered Nov 26 '18 at 14:00
Serge BallestaSerge Ballesta
78.1k957130
78.1k957130
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53478445%2fhow-can-i-override-a-setuptools-command-from-a-setup-require-dependancy%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I use the
build_py
step and not thebuild_ext
because I want to be able to produce a platform and version independant wheel. It makes sense because GNU gettext mo files can be used on any platform whatever the endianness.– Serge Ballesta
Nov 26 '18 at 10:04
From the code you provided, it's not clear why you have to put the custom command in a separate package instead of keeping it in the setup script. As you already discovered, the real issue is that you need to bootstrap the package in order to build or install it. Bootstrapping is a fragile solution - can't you isolate the build code in the setup script instead?
– hoefling
Nov 26 '18 at 20:13
@hoefling: Because the po to mo compiler is expected to be a package that can be used in other packages. Of course I had included everything in the same package in my very first tests.
– Serge Ballesta
Nov 26 '18 at 22:42