Roadmap for LUA support in GRUB

[Roman Shaposhnik]

Hi!

I was very excited to see Ubuntu 9.10 being one of the first distributions to
officially switch to GRUB v2, but there was one fly in that ointment
of happiness -- the lack of Lua scripting :-(

Browsing the archives of grub-devel reveals that Lua support was moved to
grub-extras which makes me ask these two questions:
1. Was the decision to move Lua based exclusively on the licensing concerns?
2. Is there any hope of ever seeing GRUB v2 + Lua in major distributions
once they start adopting GRUB v2 as a default boot loader?

I would appreciate all the comments!

Thanks,
Roman.

[Felix Zielcke]

Post by Roman Shaposhnik
Browsing the archives of grub-devel reveals that Lua support was moved to
1. Was the decision to move Lua based exclusively on the licensing concerns?

I don't think it was exclusively only decided because of the license,
but this is the top reason for it. License alone is the only reason why
grub-extras has been created.
But Robert or maybe Vladimir may be able to tell you more.

Post by Roman Shaposhnik
2. Is there any hope of ever seeing GRUB v2 + Lua in major distributions
once they start adopting GRUB v2 as a default boot loader?

Convince the major distributions that integrating lua in their builds is
a good reason.
In Debian we still dropped lua even though we already include
grub-extras into our build system since it was created.
In our opinion lua can be useful for a small amount of persons but is
just a waste for the majority.
lua.mod IIRC was 99K big and it was always included into the floppy
rescue images.

[Roman Shaposhnik]

Hi!

Post by Felix Zielcke

Post by Roman Shaposhnik
Browsing the archives of grub-devel reveals that Lua support was moved to
    1. Was the decision to move Lua based exclusively on the licensing concerns?

I don't think it was exclusively only decided because of the license,
but this is the top reason for it.

I'd appreciated knowing non-licensing reasons as well.

On the licensing front, though, what was an actual issue there?
After all, Lua has a respectable FOSS license and I'm sure there's tons of
MIT-licensed software in Debian. What made Lua different?

Post by Felix Zielcke

Post by Roman Shaposhnik
    2. Is there any hope of ever seeing GRUB v2 + Lua in major distributions
        once they start adopting GRUB v2 as a default boot loader?

Convince the major distributions that integrating lua in their builds is
a good reason.

That, of course, always works -- but I was hoping to find out why maintainers
feel unconvinced ;-)

Post by Felix Zielcke
lua.mod IIRC was 99K big and it was always included into the floppy
rescue images.

99K doesn't seem to be a lot compared to other auxiliary modules I find
in /boot/grub on my Ubuntu.

To some extent, that's exactly the reason of my frustration -- saving 99K
on a partition full of all sorts of stuff hardly justifies withholding a useful
feature. Don't take it the wrong way, though, this frustration is totally
misplaced on grub-devel, but to some extent had Lua module been part
of the core GRUB v2 I'm pretty sure distribution maintainers wouldn't have
thrown it out.

Thanks,
Roman.

[Vladimir 'phcoder' Serbinenko]

Post by Roman Shaposhnik
Hi!

Post by Felix Zielcke

Post by Roman Shaposhnik
Browsing the archives of grub-devel reveals that Lua support was moved to
1. Was the decision to move Lua based exclusively on the licensing concerns?

I don't think it was exclusively only decided because of the license,
but this is the top reason for it.

I'd appreciated knowing non-licensing reasons as well.

The only other reason was to encourage developpement of sh-like scripting.

Post by Roman Shaposhnik
On the licensing front, though, what was an actual issue there?
After all, Lua has a respectable FOSS license and I'm sure there's tons of
MIT-licensed software in Debian. What made Lua different?

GNU isn't Debian. GNU has a very strong copyright policy and for code to
become GNU it usually has to be copyright-assigned to FSF. It's not the
case of LUA. So we created grub-extras specifically to hold code which
is perfectly legal, free and GPLv3-compatible but not suitable for GNU.
Any distribution which doesn't have anything against code not being
copyrighted by FSF shouldn't have any reason not to add grub-extras

[Roman Shaposhnik]

Hi!

On Wed, Nov 11, 2009 at 10:53 AM, Vladimir 'phcoder' Serbinenko

Post by Vladimir 'phcoder' Serbinenko

Post by Roman Shaposhnik
I'd appreciated knowing non-licensing reasons as well.

The only other reason was to encourage developpement of sh-like scripting.

Fair enough. Would it be, then, fair to say that Lua was never meant to be
a scripting engine for GRUB, but was more of an oddity?

I was pretty excited when it first made its way into GRUB simply because it is
such a delight to do scripting in. Don't get me wrong, but compared to Lua
sh-like scripting feels like hard labor :-(

Post by Vladimir 'phcoder' Serbinenko

Post by Roman Shaposhnik
On the licensing front, though, what was an actual issue there?
After all, Lua has a respectable FOSS license and I'm sure there's tons of
MIT-licensed software in Debian. What made Lua different?

GNU isn't Debian. GNU has a very strong copyright policy and for code to
become GNU it usually has to be copyright-assigned to FSF. It's not the
case of LUA. So we created grub-extras specifically to hold code which
is perfectly legal, free and GPLv3-compatible but not suitable for GNU.
Any distribution which doesn't have anything against code not being
copyrighted by FSF shouldn't have any reason not to add grub-extras

Got it! This makes perfect sense.

Thanks,
Roman.

[Felix Zielcke]

Post by Roman Shaposhnik
Hi!

Post by Felix Zielcke

Post by Roman Shaposhnik
Browsing the archives of grub-devel reveals that Lua support was moved to
1. Was the decision to move Lua based exclusively on the licensing concerns?

I don't think it was exclusively only decided because of the license,
but this is the top reason for it.

I'd appreciated knowing non-licensing reasons as well.

Uhm actually I should have written `I don't know if it was*'
and I forgot that Robert even sent a mail to this list about it:
http://lists.gnu.org/archive/html/grub-devel/2009-09/msg00424.html

Post by Roman Shaposhnik
On the licensing front, though, what was an actual issue there?
After all, Lua has a respectable FOSS license and I'm sure there's tons of
MIT-licensed software in Debian. What made Lua different?

The difference is that GRUB is a GNU project and not some other random
open source package.
FSF wants to have the copyright of all code which is in GNU so that they
have the right to enforce the licences of all GNU projects in courts.

Post by Roman Shaposhnik

Post by Felix Zielcke
lua.mod IIRC was 99K big and it was always included into the floppy
rescue images.

99K doesn't seem to be a lot compared to other auxiliary modules I find
in /boot/grub on my Ubuntu.
To some extent, that's exactly the reason of my frustration -- saving 99K
on a partition full of all sorts of stuff hardly justifies withholding a useful
feature. Don't take it the wrong way, though, this frustration is totally
misplaced on grub-devel, but to some extent had Lua module been part
of the core GRUB v2 I'm pretty sure distribution maintainers wouldn't have
thrown it out.

Well another solution would be if you can convince lua developers to
assign copyright to the FSF for all the code we need to have in GRUB in
order to support it.
But before you go this route please talk with Robert if he's willing to
reintrage it after assigned copyright. Not that you waste your time.

[Robert Millan]

Post by Roman Shaposhnik
Hi!
I was very excited to see Ubuntu 9.10 being one of the first distributions to
officially switch to GRUB v2, but there was one fly in that ointment
of happiness -- the lack of Lua scripting :-(
Browsing the archives of grub-devel reveals that Lua support was moved to
1. Was the decision to move Lua based exclusively on the licensing concerns?

Hi,

Felix and Vladimir addressed some of the questions (size concerns, and the
risk to lose focus on our native scripting engine). There are some things
I should clarify though:

First of all, there's no license problem. We usually write our own code, but
when we have specific reasons to import it from another project, any license
that is compatible with GPL (v3 and later) would be considered suitable.

However, we only import code from external projects when there's an important
reason to do so. For example, we imported LZMA code because we needed the
best compression around, and we didn't want to reinvent the wheel. In the
specific case of LUA, this compromise didn't make sense to us since we already
had a scripting engine.

When we import external code, we don't ask that project to sign any paperwork
The GNU project considers it unpolite to make such requests to people who
didn't submit code to us in first place (this is documented in GCS).

The usual contributory agreement covers copyright assignment (so that FSF can
ensure code will always stay free), but MORE IMPORTANT than that is legal
assertion that the code you're submitting is your own. The reason we do this
is so that we can assure users and distributors that GRUB is a legally safe
codebase, and that they won't be affected by claims of copyright infringement
if they implement it.

[Roman Shaposhnik]

First of all, there's no license problem.  We usually write our own code, but
when we have specific reasons to import it from another project, any license
that is compatible with GPL (v3 and later) would be considered suitable.

Aha! So the Lua license really is a red herring here..

However, we only import code from external projects when there's an important
reason to do so.  For example, we imported LZMA code because we needed the
best compression around, and we didn't want to reinvent the wheel.  In the
specific case of LUA, this compromise didn't make sense to us since we already
had a scripting engine.

...the real reason seems to be that you don't really believe in Lua as a primary
scripting language for GRUB, correct?

IOW, the inclusion of Lua was more of a fluke than a deliberate
decision of investing
in a different scripting engine.

This is not a value judgment -- just an attempt to figure things out.
Personally I'm really
excited about Lua. I see that a lot folks share my opinion (I can't
wait to see Lua as
a scripting engine for NetBSD kernel, for example) and I thought that
GRUB project
was among these folks.

Thanks,
Roman.

[Felix Zielcke]

Post by Roman Shaposhnik

Post by Robert Millan
First of all, there's no license problem. We usually write our own code, but
when we have specific reasons to import it from another project, any license
that is compatible with GPL (v3 and later) would be considered suitable.

Aha! So the Lua license really is a red herring here..

My bad I used the wrong word, I should have said s/license/copyright/
But anyway

Post by Roman Shaposhnik

Post by Robert Millan
However, we only import code from external projects when there's an important
reason to do so. For example, we imported LZMA code because we needed the
best compression around, and we didn't want to reinvent the wheel. In the
specific case of LUA, this compromise didn't make sense to us since we already
had a scripting engine.

...the real reason seems to be that you don't really believe in Lua as a primary
scripting language for GRUB, correct?

Why do you think Lua as a *primary* language/format/whatever for
grub.cfg would be right?
With sh grub.cfg isn't that different from any other normal config file
or even GRUB Legacy's menu.lst
Learning Lua might be not so difficult for average experienced Linux
users.
But things are moving. Not every average Linux user has now some
experience.

[Vladimir 'phcoder' Serbinenko]

Post by Roman Shaposhnik

Post by Robert Millan
First of all, there's no license problem. We usually write our own code, but
when we have specific reasons to import it from another project, any license
that is compatible with GPL (v3 and later) would be considered suitable.

Aha! So the Lua license really is a red herring here..

We already explained the reasons. Grub-extras is for code which is for
imported code ehich doesn't need to reside in main trunk to be
functional. ZFS is a similar case. On the other hand LZMA needs to be in
main trunk. Occasionally exceptions may occur but it doesn't discard
general rule.

Post by Roman Shaposhnik

Post by Robert Millan
However, we only import code from external projects when there's an important
reason to do so. For example, we imported LZMA code because we needed the
best compression around, and we didn't want to reinvent the wheel. In the
specific case of LUA, this compromise didn't make sense to us since we already
had a scripting engine.

...the real reason seems to be that you don't really believe in Lua as a primary
scripting language for GRUB, correct?

Please don't become personal on this. Except rescue parsers all parsers
are implemented the same way and can replace one another. So what do you
mean by "primary scripting language"? If what you mean is just one being
default then no project can make default which suit all usres. It's what
configuration is for. If you speak about invested efforts then you can't
force people to spend time on something they don't want (but you can pay
one of us to do stuff you need).

Post by Roman Shaposhnik
IOW, the inclusion of Lua was more of a fluke than a deliberate
decision of investing
in a different scripting engine.
This is not a value judgment -- just an attempt to figure things out.
Personally I'm really
excited about Lua. I see that a lot folks share my opinion (I can't
wait to see Lua as
a scripting engine for NetBSD kernel, for example) and I thought that
GRUB project
was among these folks.

As I said you can't ask everyone to share your language preferences. But
patches are of course welcome

Post by Roman Shaposhnik
Thanks,
Roman.
_______________________________________________
Grub-devel mailing list
http://lists.gnu.org/mailman/listinfo/grub-devel

[Roman Shaposhnik]

On Fri, Nov 13, 2009 at 3:54 AM, Vladimir 'phcoder' Serbinenko

Post by Vladimir 'phcoder' Serbinenko

Post by Roman Shaposhnik
Aha! So the Lua license really is a red herring here..

We already explained the reasons.

You did. But, unfortunately, your explanation was not entirely correct. Which
is fine, because thanks to Robert I now know the answer that actually
makes perfect sense.

Licensing is thorny issues. It usually helps when it gets explained very, very
precisely.

Post by Vladimir 'phcoder' Serbinenko
Please don't become personal on this.

What would make you say this? I don't think I've made any statements that
could be interpreted like making this matter a personal one.

Post by Vladimir 'phcoder' Serbinenko
Except rescue parsers all parsers
are implemented the same way and can replace one another. So what do you
mean by "primary scripting language"?

A scripting language that is actively maintained and used for writing extensions
to GRUB. My original understanding was that Lua would fit this description,
now it seems that it was more like an odd experiment in GRUB trunk.

IOW, at the moment -- if I need to script GRUB my only portable choice would be,
unfortunately, its internal shell. With the current state of things I
can't rely on
Lua always being there for me.

Post by Vladimir 'phcoder' Serbinenko
If what you mean is just one being
default then no project can make default which suit all usres. It's what
configuration is for. If you speak about invested efforts then you can't
force people to spend time on something they don't want (but you can pay
one of us to do stuff you need).

There's no need to be upset. What I'm after here is a simple statement of fact,
not value judgments. Its ok for GRUB not be interested in Lua. As long
as everybody
is clear on that -- no harm done.

Thanks,
Roman.

[Bean]

Post by Roman Shaposhnik
A scripting language that is actively maintained and used for writing extensions
to GRUB. My original understanding was that Lua would fit this description,
now it seems that it was more like an odd experiment in GRUB trunk.
IOW, at the moment -- if I need to script GRUB my only portable choice would be,
unfortunately, its internal shell. With the current state of things I
can't rely on
Lua always being there for me.

Hi,

I like the LUA language, actually It's me who add it in the first
place. Some of the purpose is detection of OS items at runtime, and
interaction with graphic menu system. And honestly, I'm not happy
with it being removed from main repository as this make it harder to
maintain.

But no worry, I've created a fork project BURG that contains some
brand-new features, the LUA engine will be added back soon.

[Roman Shaposhnik]

Post by Bean

Post by Roman Shaposhnik
A scripting language that is actively maintained and used for writing extensions
to GRUB. My original understanding was that Lua would fit this description,
now it seems that it was more like an odd experiment in GRUB trunk.
IOW, at the moment -- if I need to script GRUB my only portable choice would be,
unfortunately, its internal shell. With the current state of things I
can't rely on
Lua always being there for me.

Hi,
I like the LUA language, actually It's me who add it in the first
place. Some of  the purpose is detection of OS items at runtime, and
interaction with graphic menu system.  And honestly, I'm not happy
with it being removed from main repository as this make it harder to
maintain.
But no worry, I've created a fork project BURG that contains some
brand-new features, the LUA engine will be added back soon.

Perfect! Thanks a million for chiming in -- that's exactly the kind of
information I was looking for.

Thanks,
Roman (switching from grub-devel to burg-devel...)

[Robert Millan]

Post by Bean
But no worry, I've created a fork project BURG that contains some
brand-new features, the LUA engine will be added back soon.

Does that make it any easier than maintaining LUA in grub-extras?

When moving it there, I made it clear [1] that anyone who wishes to work
on LUA is welcome to use the grub-extras repository for that purpose [1].

I expected a reply from you, but there was none.

[1] http://lists.gnu.org/archive/html/grub-devel/2009-09/msg00424.html

[Bean]

Post by Robert Millan

Post by Bean
But no worry, I've created a fork project BURG that contains some
brand-new features, the LUA engine will be added back soon.

Does that make it any easier than maintaining LUA in grub-extras?
When moving it there, I made it clear [1] that anyone who wishes to work
on LUA is welcome to use the grub-extras repository for that purpose [1].
I expected a reply from you, but there was none.
[1] http://lists.gnu.org/archive/html/grub-devel/2009-09/msg00424.html

Hi,

As I see it, one problem of grub-extra is that it can't be compiled
separately, so user have to setup an environment to build it, which is
not straightforward. And as they have two revision system, this make
it difficult to track previous bug. For example, it's hard to tell
which revision of grub-extra can compile with which main stream
revision. And it's also the question of confidence, I don't see anyone
claim responsibility for grub-extra, it' more like a garbage dump to
me. It's hard to convince others to send patches on something that's
not actively maintained.

[Vladimir 'phcoder' Serbinenko]

Post by Bean

Post by Robert Millan

Post by Bean
But no worry, I've created a fork project BURG that contains some
brand-new features, the LUA engine will be added back soon.

Does that make it any easier than maintaining LUA in grub-extras?
When moving it there, I made it clear [1] that anyone who wishes to work
on LUA is welcome to use the grub-extras repository for that purpose [1].
I expected a reply from you, but there was none.
[1] http://lists.gnu.org/archive/html/grub-devel/2009-09/msg00424.html

Hi,
As I see it, one problem of grub-extra is that it can't be compiled
separately, so user have to setup an environment to build it, which is
not straightforward.

Then propose a better system. I don't find any problem with setting
GRUB_CONTRIB

Post by Bean
And as they have two revision system, this make
it difficult to track previous bug. For example, it's hard to tell
which revision of grub-extra can compile with which main stream
revision.

It's always latest-to-latest. And it's how it's done in Debian.

Post by Bean
And it's also the question of confidence, I don't see anyone
claim responsibility for grub-extra, it' more like a garbage dump to
me. It's hard to convince others to send patches on something that's
not actively maintained.

The same code wouldn't have recieved more attention if it was in the
mainline.
You can pretty much say nobody looks into hello/hello.c but is it a
reason to tell it garbage.
I personally handle grub-extras the same way as GRUB2
Actually the same people that take care of mainline also take care of
grub-extras. I also have two new things on the queue for grub-extras:
LUKS and gPXE import.

[Bean]

On Sat, Nov 14, 2009 at 4:22 AM, Vladimir 'phcoder' Serbinenko

Post by Vladimir 'phcoder' Serbinenko

Post by Bean
And as they have two revision system, this make
it difficult to track previous bug. For example, it's hard to tell
which revision of grub-extra can compile with which main stream
revision.

It's always latest-to-latest. And it's how it's done in Debian.

Hi,

Consider this problem. I've made some patch on LUA that uses
experimental feature. I can't commit it to grub-extra as it would
break the building of grub-pc, I can't commit to my branch either as
LUA is not in it, so where do I commit the patch ?

[Vladimir 'phcoder' Serbinenko]

Post by Bean
On Sat, Nov 14, 2009 at 4:22 AM, Vladimir 'phcoder' Serbinenko

Post by Vladimir 'phcoder' Serbinenko

Post by Bean
And as they have two revision system, this make
it difficult to track previous bug. For example, it's hard to tell
which revision of grub-extra can compile with which main stream
revision.

It's always latest-to-latest. And it's how it's done in Debian.

Hi,
Consider this problem. I've made some patch on LUA that uses
experimental feature. I can't commit it to grub-extra as it would
break the building of grub-pc, I can't commit to my branch either as
LUA is not in it, so where do I commit the patch ?

We already discussed this problem and solution was straightforward: we
migrate grub-extras to bazaar too and we create experimental branches
there too

[Robert Millan]

Post by Bean
Hi,
As I see it, one problem of grub-extra is that it can't be compiled
separately, so user have to setup an environment to build it,

Why? Modules from grub-extras can be compiled in-tree just as fine.

Post by Bean
And as they have two revision system, this make
it difficult to track previous bug.

I've been looking into migrating grub-extras to Bazaar, but haven't found
the time. Would it help if I do this?

Post by Bean
And it's also the question of confidence, I don't see anyone
claim responsibility for grub-extra, it' more like a garbage dump to
me. It's hard to convince others to send patches on something that's
not actively maintained.

I think you're presenting the problem in a way that can't have any possible
solution:

- I made an open offer to anyone who would want to maintain LUA in
grub-extras. You weren't interested.

- Now you say that you made a fork of GRUB because LUA in grub-extras
isn't being properly maintained.

I'm afraid I can't help you there. If you have other reasons for this, feel
free to explain them, and maybe we can find a solution.

[Bruce O. Benson]

Post by Bean
But no worry, I've created a fork project BURG that contains some
brand-new features, the LUA engine will be added back soon.

Is there a URL for BURG yet?
As soon as I see a bootloader that uses Lua as its scripting/config engine,
I'm switching to it.


Thanks,
Bruce.

[Bruce O. Benson]

Post by Bruce O. Benson

Post by Bruce O. Benson
As soon as I see a bootloader that uses Lua as its scripting/config

engine,

Post by Bruce O. Benson
I'm switching to it.

We have LUA support for GRUB. It's in grub-extras.

Exactly. I didn't say 'support'. I said 'uses'.

[Robert Millan]

Post by Roman Shaposhnik

Post by Robert Millan
However, we only import code from external projects when there's an important
reason to do so.  For example, we imported LZMA code because we needed the
best compression around, and we didn't want to reinvent the wheel.  In the
specific case of LUA, this compromise didn't make sense to us since we already
had a scripting engine.

...the real reason seems to be that you don't really believe in Lua as a primary
scripting language for GRUB, correct?

No. It's quoted above in this mail. Please read it again...