I'm using Modelio.getInstance().getScriptService().getScriptEngine(None) to retrieve script so to display (print) messages in the script window of Modelio.
You may capture the output of the script with a java code like this:
ScriptEngine engine = Modelio.getInstance().getScriptService().getScriptEngine(getClass().getClassLoader());
StringWriter out = new StringWriter();
engine.getContext().setWriter(out);
engine.eval("print 'Hello world!'");
// Get the result with
out.toString();
You may then create a SWT dialog to display the output.