going on. You see, when using Java and HM.invokeWithArguments(),
conversions (varargs included) are taking place at two levels. The 1st
it invokes the target method. The 1st is specified by JLS, while the 2nd
is specified in the javadoc of .invokeWithArguments.
MH.invokeWithArguments() (and MH.invoke()). First thing you have to know
way of producing MethodHandle(s), but its all the same. You have to be
they are varargs collectors. When invoking those methods, they accept
array argument.
This does not happen when you invoke such methods using reflection.
lookup / unreflect with MH.asFixedArity() method. If the method handle
collection of trailing positional arguments into an array argument.
as using Method.invoke().
Post by Rony G. Flatscherthank you very much for your thorough and extensive analysis taking
the evolvement of Java into account!
You use the Java compiler and its behaviour (which adds syntax sugar
to the Java language and is known to stick strictly to the Java
language specification) for explanations and as reference, which is
also what I have been doing in the past to counter-check the
implemented behaviour of the bridge.
The Java compiler has a very different outset: *at compilation time*
it knows everything it needs to know for compiling. If something is
missing or wrong at compilation time it is able to stop compilation
raising appropriate errors, thereby inhibiting the successful creation
of Java classes as long as the Java programmer does not correct
erroneous code or supplies missing vital information. Java being a
strictly typed language allows one to cast arguments to force the
compiler to pick the method the Java programmer had in mind in the
case that the compiler could choose one from a set of a signatures.
By contrast a bridge for an interpreted, dynamically typed language
(like ooRexx) needs to reflect and infer at *runtime* in order to be
able to choose the Java methods with the appropriate signature. In
this particular case all primitive datatypes are (ooRexx/C) strings in
the dynamically typed language such that at runtime each argument
needs to be checked whether they contain valid values for conversion
to primitive datatypes in the signature of a Java method candidate.
In the dynamically typed language there is no means available for
casting that would be needed to help solve the presented problem
(there is optionally a box()-routine in the bridge that allows the
ooRexx programmer to explicitly determine which primitive type is to
be used for picking an appropriate Java method).
The new code in the bridge (had to rewrite that part from scratch
because of Java 9 and later) uses java.lang.reflect to pick an
appropriate Constructor, Method or Field object. For Java 1.6 and 1.7
(although java.lang.invoke/MethodHandle got introduced with 1.7 I
found cases where inaccessible types did not have accessible ancestors
in the JRE) java.lang.reflect invocation will be carried out, for Java
1.8 and later java.lang.invoke/MethodHandle invocations gets employed.
Comparing the speed between invocations using java.lang.reflect and
java.lang.invoke, java.lang.invoke wins, but by a quite small margin.
Upon further testunit tests the rewritten bridge using
java.lang.invoke would execute all ooRexx programs like
java.lang.reflect since Java 1.1 with one single exception so far
which I reported here, where invocation via java.lang.reflect behaves
differently to java.lang.invoke.
Because of this discrepancy the question would be: is this considered
a bug that will be corrected? Would there be otherwise a solution
possible in the bridge that could take care of situations like this,
where the bridge has no context-information other than referring to
Constructor, Method or Field objects with the supplied arguments?
* if the unreflect() method gets used then MH.invoke() should behave
like in the java.lang.reflect case; this means that the same
assumptions should govern MH.invoke(): reason being that if a
java.lang.reflect object gets unreflected, also the established
java.lang.reflect rules should keep working in this case for the
MH.invoke* in order to remain fully compatible with each other,
o if a MH gets created without unreflect() then this would not be necessary
* the bridge could restrain itself to only use java.lang.reflect for
invocations on Java 8 and higher instead of java.lang.invoke
o clearly, the current Java development efforts in this corner
are concentrated on java.lang.invoke, so it would be desirable
to switch from java.lang.reflect to java.lang.invoke which is
not possible as long as this problem (different behaviour in
j.l.r vs. j.l.i) persists
* do not fix this, but document this as an incompatibility with the
ramification that existing programs in dynamically typed languages
need to be rewritten (e.g. in this case creating an Array object
from the List and supply that instead)
* ... ?
---rony
P.S.: It would be great if Java implemented reflective invocations for
dynamically typed languages as this would solve the problem for all
such languages once and forever in a standard manner, rather than have
every bridge implementor create his own implementation, which is
challenging, effortful and possibly error prone. In my case the
assumptions that governed the original implementation for reflective
Constructor, Method and Field access for over fifteen years got broken
by Java 9, because j.l.r (notable 'setAccessible') changed in a
fundamental aspect. If Java could be emplyoed instead, the Java
supplied implementation would be able to adopt changes in the inner
workings accordingly thereby insulating bridges from such details and
problems.
Post by Peter LevartI think what you found is a discrepancy between handling of varargs
methods using reflection vs. method handle's .invokeWithArguments().
Reflection basically ignores the fact that some methods are varargs
methods. It treats them exactly the same as if they had an array
parameter (not a varargs parameter).
   public static <T> List<T> asList(T ... elements)
   public static <T> List<T> asList(T[] elements)
That's because varargs were introduced to Java in Java 5 as a kind of
compilation sugar, implemented as an array parameter, while
reflection is an older beast and wasn't updated to behave differently
with varargs methods.
MethodHandle(s) came later, in Java 7, and took into consideration varargs methods.
       Method asListMethod = Arrays.class.getMethod("asList",
Object[].class);
       String[] elements = {"a", "b", "c"};
       System.out.println(
           asListMethod.invoke(null, elements)
       );
You get: IllegalArgumentException: wrong number of arguments
   public Object invoke(Object obj, Object... args)
... so java compiler (in order to be backwards source compatible with
pre-varargs methods) treats this invocation in a way that just passes
the String[] elements to the invoke method via the args parameter
without any conversion (because String[] is a subtype of Object[]).
Mathod.invoke then treats elements of the args[] array as individual
verbatim parameters to be passed to the method when invoking it. As
reflection treats Arrays.asList() as a normal no-varargs method which
accepts a single Objet[] parameter and args[] contains 3 elements to
be passed to a method with 3 parameters, exception occurs.
       Method asListMethod = Arrays.class.getMethod("asList",
Object[].class);
       MethodHandle asListMH =
MethodHandles.lookup().unreflect(asListMethod);
       String[] elements = {"a", "b", "c"};
       System.out.println(
           asListMH.invokeWithArguments(elements)
       );
   public Object invokeWithArguments(Object... arguments)
... so java compiler (in order to be backwards source compatible with
pre-varargs methods) treats this invocation in a way that just passes
the String[] elements to the invokeWithArguments method via the
single 'arguments' parameter without any conversion (because String[]
is a subtype of Object[]). MethodHandle.invokeWithArguments therefore
takes 3 array elements and tries to invoke the underlying asList
method with them. It observes that Arrays.asList is a varargs method,
so it wraps the 3 "wannabe parameters" with the Object[] and passes
the array to the asList method as single parameter. The result of
   [a, b, c]
If you want MethodHandle.invokeWithArguments() to treat "wannabe
parameters" passed to it as verbatim parameters regardless of the
variable/fixed arity of the method to be invoked, then you can
       Method asListMethod = Arrays.class.getMethod("asList",
Object[].class);
       String[] elements = {"a", "b", "c"};
       System.out.println(
           asListMethod.invoke(null, (Object) elements)
       );
       MethodHandle asListMH =
MethodHandles.lookup().unreflect(asListMethod);
       asListMH = asListMH.asFixedArity();
       System.out.println(
           asListMH.invokeWithArguments((Object) elements)
       );
   [a, b, c]
   [a, b, c]
What happens here is the following (will only describe the MH case -
   public Object invokeWithArguments(Object... arguments)
... so java compiler this time wraps the '(Object) elements' value
with an Object[] because the value is of Object type (which is not a
subtype of Object[]) - no backwards compatibility needed here,
varargs conversion kicks in in the javac.
MethodHandle.invokeWithArguments therefore takes an array with 1
element (the element being 'elements' array) and tries to invoke the
underlying asList method with it. This time it observes that the
method is not a varargs method, (because MethodHandle was transformed
to fixed arity method handle) so it passes the single 'elements'
wannabee parameter to the Arrays.asList single Object[] parameter -
this invocation works because fixed arity asList() takes Object[] and
'elements' is a String[]...
Hope this helps you understand what's going on.
Regards, Peter
Post by Rony G. FlatscherPost by Rony G. FlatscherWell, still trying to find out what the reason is, that core
reflection's invoke behaves differently to MethodHandle's
invokeWithArguments in one single case so far (using the method
java.utli.Arrays.asList(...)).
Here is a little Java program that excercises reflective access to
"java.util.Arrays.asListâ(T... a)" using core reflection, i.e.
"Method.invokeâ(Object obj, Object... args)" and
import java.util.*;
import java.lang.reflect.*;
import java.lang.invoke.*;
class DemoAsListProblem
{
public static void main (String args[])
{
String arrNames[]=new String[] { "anne", "bert", "celine"};
System.out.println("[1] (main) arrNames=\""+arrNames+"\", .toString()=\""+arrNames.toString()+"\"");
List listNames=Arrays.asList(arrNames);
System.out.println("[2] (main) after invoking Arrays.asList(arrNames), listNames=\""+listNames+"\"");
System.out.println("\ninvoking testReflective() ...\n");
testReflective();
}
public static void testReflective()
{
String arrNames[]=new String[] { "anne", "bert", "celine"};
System.out.println("[3] (testReflective) arrNames=\""+arrNames+"\", .toString()=\""+arrNames.toString()+"\"");
Method methAsList=null;
List listNames=null;
try {
Class paramTypes[]=new Class[] { (new Object[0]).getClass() };
methAsList=Arrays.class.getDeclaredMethod("asList", paramTypes);
System.out.println("--- (core reflection) Method object asList: "+methAsList);
System.out.println("\n--- (core reflection) now invoking Arrays.asList() via Method.invoke(...):");
listNames=(List) methAsList.invoke( null, (Object[]) new Object[]{arrNames} ); // static method
System.out.println("[4a] --- (CR) methAsList.invoke( null, (Object[]) new Object[]{arrNames} ) -> listNames=\""+listNames+"\"");
listNames=(List) methAsList.invoke( null, (Object) new Object[]{arrNames} ); // static method
System.out.println("[4b] --- (CR) methAsList.invoke( null, (Object) new Object[]{arrNames} ) -> listNames=\""+listNames+"\"");
// listNames=(List) methAsList.invoke( null, (Object[]) arrNames ); // static method
// System.out.println("[5a] --- (CR) methAsList.invoke( null, (Object[]) arrNames ) -> listNames=\""+listNames+"\"");
listNames=(List) methAsList.invoke( null, (Object) arrNames ); // static method
System.out.println("[5b] --- (CR) methAsList.invoke( null, (Object) arrNames ) -> listNames=\""+listNames+"\"");
}
catch (Throwable t)
{
System.err.println("oops #1: "+t);
t.printStackTrace();
System.exit(-1);
}
System.out.println("\n--- (MH) now invoking Arrays.asList() via MethodHandle.invokeWithArguments(...):");
MethodHandles.Lookup lookup=MethodHandles.lookup();
MethodHandle mh=null;
try {
mh=lookup.unreflect(methAsList);
System.out.println("--- (MH) unreflected MethodHandle mh: "+mh);
listNames=(List) mh.invokeWithArguments( (Object[]) new Object[]{arrNames} );
System.out.println("[6a] --- (MH) mh.invokeWithArguments( (Object[]) new Object[]{arrNames} ) -> listNames=\""+listNames+"\"");
listNames=(List) mh.invokeWithArguments( (Object) new Object[]{arrNames} );
System.out.println("[6b] --- (MH) mh.invokeWithArguments( (Object) new Object[]{arrNames} ) -> listNames=\""+listNames+"\"");
listNames=(List) mh.invokeWithArguments( (Object[]) arrNames );
System.out.println("[7a] --- (MH) mh.invokeWithArguments( (Object[]) arrNames ) -> listNames=\""+listNames+"\"");
listNames=(List) mh.invokeWithArguments( (Object) arrNames );
System.out.println("[7b] --- (MH) mh.invokeWithArguments( (Object) arrNames ) -> listNames=\""+listNames+"\"");
}
catch (Throwable t)
{
System.err.println("oops #2: "+t);
t.printStackTrace();
System.exit(-2);
}
}
}
[2] (main) after invoking Arrays.asList(arrNames), listNames="[anne, bert, celine]"
invoking testReflective() ...
--- (core reflection) Method object asList: public static java.util.List java.util.Arrays.asList(java.lang.Object[])
[4a] --- (CR) methAsList.invoke( null, (Object[]) new Object[]{arrNames} ) -> listNames="[anne, bert, celine]"
[5b] --- (CR) methAsList.invoke( null, (Object) arrNames ) -> listNames="[anne, bert, celine]"
--- (MH) unreflected MethodHandle mh: MethodHandle(Object[])List
[7a] --- (MH) mh.invokeWithArguments( (Object[]) arrNames ) -> listNames="[anne, bert, celine]"
So a String array is turned into a List using
Arrays.asList(strArray). Doing it with core reflection yields
different results to doing it with invokeWithArguments().
I would have expected that [4a] and [6a] would behave the same.
Using reflective invoke() and invokeWithArguments() has been
working for my bridge for all the test units (using literally the
same arguments in both cases) interchangeably, the one exception is
Arrays.asList().
Maybe I am not seeing the obvious (having done too much "close-up
tests" for quite some time now). So any hint, insight, help would
be really appreciated!
---rony
As a few months have gone by without any follow-ups, I have been
wondering whether there is a solution at all, if this is a problem
rooted in the current implementation of MethodHandle methods not
being 100% compatible with reflective invoke (which may imply that
one needs to stick to use reflective invoke and forgo MethodHandle
invokes for good).
---rony