If somebody has time to try this, I'd love to hear from you!

I was also thinking about yaegi too. Its amazing to have an interpreter.

I am thinking of using it for the CI scripts, so that i dont have to use the providers scripting language, but instead just write the scripts in golang.

So the first thing the CI does is install yaegi, and then your cooking...

I am using Azure pipelines so that i can run the tests and builds in all Operating systems.

Great! I'd love to see some examples of what you're doing—we could add them to the README to show how to use script in Ci pipelines.

Ok will buzz you when i have some ready

Hey @joeblew99, how are you getting on? Any news?

@bitfield Got nothing done on it. Had to work on quic networking stack. Sorry but you know how it is with development - always so many things going on.

I think that its still worth while for CI.

No problem, maybe somebody else will be able to pick this up and have a play with it!

Hello,

I tried to play with Yaegi + script today. I did not dig super far but when trying this fairly simple program:

package main

import (
    "fmt"
    "log"
    "github.com/bitfield/script"
)

func main() {
    p, err := script.Echo("test").String()
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(p)    
}

I have no issue with go run main.go while I have the following panic using yaegi main.go.

lobre@nixos /t/tmp.0DmEYWtsjJ> yaegi main.go
/home/lobre/Lab/go/src/github.com/bitfield/script/read_auto_closer.go:31:5: panic
/home/lobre/Lab/go/src/github.com/bitfield/script/read_auto_closer.go:18:5: panic
/home/lobre/Lab/go/src/github.com/bitfield/script/sinks.go:22:5: panic
/home/lobre/Lab/go/src/github.com/bitfield/script/sinks.go:66:15: panic
main.go:10:15: panic
reflect.Set: value of type *strings.Reader is not assignable to type io.Closer
goroutine 1 [running]:
runtime/debug.Stack(0x1, 0xc0002dec00, 0x40)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/runtime/debug/stack.go:24 +0x9d
github.com/containous/yaegi/interp.(*Interpreter).Eval.func1(0xc0001ebde8)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/interp.go:320 +0xbb
panic(0xca0b80, 0xc000435dc0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/runtime/panic.go:679 +0x1b2
github.com/containous/yaegi/interp.runCfg.func1(0xc00029c280, 0xc0001b9600)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:113 +0x20c
panic(0xca0b80, 0xc000435dc0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/runtime/panic.go:679 +0x1b2
github.com/containous/yaegi/interp.runCfg.func1(0xc00029c6e0, 0xc0003bd000)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:113 +0x20c
panic(0xca0b80, 0xc000435dc0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/runtime/panic.go:679 +0x1b2
github.com/containous/yaegi/interp.runCfg.func1(0xc00029c820, 0xc0003b1600)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:113 +0x20c
panic(0xca0b80, 0xc000435dc0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/runtime/panic.go:679 +0x1b2
io/ioutil.readAll.func1(0xc0001eb3c0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/io/ioutil/ioutil.go:30 +0x101
panic(0xca0b80, 0xc000435dc0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/runtime/panic.go:679 +0x1b2
github.com/containous/yaegi/interp.runCfg.func1(0xc00029caa0, 0xc000394a00)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:113 +0x20c
panic(0xca0b80, 0xc000435dc0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/runtime/panic.go:679 +0x1b2
github.com/containous/yaegi/interp.runCfg.func1(0xc00029cbe0, 0xc000397b00)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:113 +0x20c
panic(0xca0b80, 0xc000435dc0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/runtime/panic.go:679 +0x1b2
reflect.Value.assignTo(0xd841a0, 0xc00045df00, 0x16, 0xddfcdf, 0xb, 0xcfbce0, 0xc000435da0, 0x1, 0xd841a0, 0xc00045df00)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/reflect/value.go:2403 +0x432
reflect.Value.Set(0xcfbce0, 0xc000435da0, 0x194, 0xd841a0, 0xc00045df00, 0x16)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/reflect/value.go:1537 +0xbd
github.com/containous/yaegi/interp.typeAssert.func1(0xc00029cbe0, 0xc00045cc80)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:162 +0xba
github.com/containous/yaegi/interp.runCfg(0xc000397b00, 0xc00029cbe0)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:119 +0x7a
github.com/containous/yaegi/interp.call.func5(0xc00029caa0, 0xc000454d90)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:787 +0xb10
github.com/containous/yaegi/interp.runCfg(0xc000394a00, 0xc00029caa0)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:119 +0x7a
github.com/containous/yaegi/interp.genFunctionWrapper.func2.1(0xc0004741c0, 0x1, 0x1, 0xc0004741c0, 0x0, 0x0)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:532 +0x3a7
reflect.callReflect(0xc00046c1e0, 0xc0001eb240, 0xc0001eb228)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/reflect/value.go:549 +0x322
reflect.makeFuncStub(0xc000476004, 0x5fc, 0x5fc, 0x600, 0x600, 0xc000476000, 0xc0001eb2e0, 0x51e1d4, 0xc00046c1e0, 0xc000476004, ...)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/reflect/asm_amd64.s:20 +0x42
github.com/containous/yaegi/stdlib._io_Reader.Read(0xc00046c1e0, 0xc000476004, 0x5fc, 0x5fc, 0x4, 0x0, 0x0)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/stdlib/go1_13_io.go:177 +0x44
bytes.(*Buffer).ReadFrom(0xc0001eb358, 0x7f81b0b408c8, 0xc00046c1e0, 0xc0001eb338, 0x40efa3, 0xcfbd60)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/bytes/buffer.go:204 +0xb4
io/ioutil.readAll(0x7f81b0b408c8, 0xc00046c1e0, 0x200, 0x0, 0x0, 0x0, 0x0, 0x0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/io/ioutil/ioutil.go:36 +0x100
io/ioutil.ReadAll(0x7f81b0b408c8, 0xc00046c1e0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/io/ioutil/ioutil.go:45 +0x3e
reflect.Value.call(0xccc4e0, 0xe11de0, 0x13, 0xdd5519, 0x4, 0xc000474040, 0x1, 0x1, 0xc00041b680, 0xc000444c00, ...)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/reflect/value.go:460 +0x5f6
reflect.Value.Call(0xccc4e0, 0xe11de0, 0x13, 0xc000474040, 0x1, 0x1, 0xc000452e58, 0xc0001eb778, 0x665f11)
	/nix/store/la9h5rrc53yd5ji79w0vlhcqdhhhb29q-go-1.13.8/share/go/src/reflect/value.go:321 +0xb4
github.com/containous/yaegi/interp.callBin.func6(0xc00029c820, 0xc000456410)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:919 +0x1a6
github.com/containous/yaegi/interp.runCfg(0xc0003b1600, 0xc00029c820)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:119 +0x7a
github.com/containous/yaegi/interp.call.func5(0xc00029c6e0, 0xc000455340)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:787 +0xb10
github.com/containous/yaegi/interp.runCfg(0xc0003bd000, 0xc00029c6e0)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:119 +0x7a
github.com/containous/yaegi/interp.call.func5(0xc00029c280, 0xc000466380)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:787 +0xb10
github.com/containous/yaegi/interp.runCfg(0xc0001b9600, 0xc00029c280)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:119 +0x7a
github.com/containous/yaegi/interp.(*Interpreter).run(0xc0002982c0, 0xc0001b8a00, 0xc00029c000)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/run.go:97 +0x239
github.com/containous/yaegi/interp.(*Interpreter).Eval(0xc0002982c0, 0xc0002a4000, 0xd2, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/lobre/Lab/go/src/github.com/containous/yaegi/interp/interp.go:388 +0x427
main.main()
	/home/lobre/Lab/go/src/github.com/containous/yaegi/cmd/yaegi/yaegi.go:94 +0x548

I am not super confortable with Yaegi's internals so I am not sure to be able to quickly debug this by now 😞

Might need a little bit more of experimentation.
Loric

Oh, great! Thanks for finding this. As far as I know this is the first real bug anyone's ever found in script, so as you can imagine, I'm delighted.

Would you mind opening this as a new issue? I'll grab Yaegi and see if I can reproduce this and figure out what's going on.

Actually, cancel that; I figured it out! Or rather, there's an oddity in the code which, though it works perfectly under standard Go, causes Yaegi to crash. That's a Yaegi issue, which I'll go ahead and file as soon as I've got a reproducing program that doesn't use script.

I'll push a change to script which avoids triggering the Yaegi bug.

I was wondering if the problem was actually on the Yaegi side! Well spotted!
And thanks for thinking about a change to avoid this behaviour, I look forward to being able to test this!

@lobre this is containous/yaegi#558.

I hadn't tried Yaegi before, so thanks very much to @lobre for suggesting it. Having had a look, my sense is that it's at a fairly early stage at the moment, and there seem to be problems using it with lots of well-known and widely-used libraries. So it's definitely worth trying Yaegi with script programs, but I wouldn't be in the least surprised if they failed to work for all sorts of reasons (one obvious thing is that Yaegi doesn't currently work outside GOPATH, so what I think of as 'modern' Go programs simply won't run at all under Yaegi).

So the issue was with the ReadAutoCloser type, which wraps an io.Reader in such a way that it will automatically be closed once it's read to completion. It's just a struct wrapper:

type ReadAutoCloser struct {
	r io.Reader
}

But the Read method is magical, because it detects io.EOF and calls Close on the reader. In order to do that, of course, we have to make sure that it actually has a Close method! So the constructor, NewReadAutoCloser(), checks this. If the supplied reader is not an io.Closer, then it is wrapped by ioutil.NopCloser, which adds a no-op Close method to any reader.

So far, so good. But note that we declared r as an io.Reader, not an io.ReadCloser! So although we have guaranteed in code that it will be a Closer, we didn't guarantee it in the type. Vanilla Go has no problem with this; why would it? When we call Close() on the object, there is a type-assertion to io.Closer which would panic if it didn't have a Close method. This panic never happens because the constructor ensures it won't.

Whatever Yaegi does, though, causes this to fail. Even though it's not strictly a bug in script (or even a bug in Yaegi, necessarily), it does expose this 'hidden' type mismatch, so I've pushed a fix for it in 9d58708. The fix is simple; just declare r as the type we have already programmatically ensured it must be:

type ReadAutoCloser struct {
	r io.ReadCloser
}

@lobre, does this fix your example program, or do you at least not get the exact same crash?

Hello @bitfield,

Thanks for this effort trying to understand and find a fix!
As you said, I think the problem might occur with the way yaegi uses the reflect package.

Unfortunately, I still have a panic with your latest fix but now with the io.ReadCloser type assertion.
For me, this fix still makes sense because it makes the struct more documented by its content. We know that this reader is closable at a glance.

I am not sure it is worth trying to make script work with yaegi anyway. As you said, it might still be in early stage and it might be better waiting for more stabilization on their side.

I am sure both projects will be a cool combo used together as soon as yaegi gets ready!

Loric

No worries, @lobre! Hopefully other people trying out Yaegi with script will come across this issue and find some useful information. I'll keep it open for now.