[Py4j-users] How to resolve "Py4jError: Trying to call a package"

[Stephen B. <ja...@gm...>]

My team has added a module for pyspark which is a heavy user of py4j.  I
have not been successful to invoke the newly added scala/java classes from
python (pyspark) via their java gateway.


The pyspark code creates a java gateway:
              gateway = JavaGateway(GatewayClient(port=gateway_port),
auto_convert=False)

Here is an example of existing (/working) pyspark java_gateway code:

    java_import(gateway.jvm, "org.apache.spark.sql.hive.HiveContext")

I have attempted to import a trivial custom class:

    java_import(gateway.jvm, "com.mycompany.spark.test.JarTest")


But when invoking I get:

   tclass = self._jvm.com.mycompany.spark.test.JarTest
    tclass.ping()   # Error happens on this line


Py4JError: Trying to call a package.


So I would request guidance / documentation on how to
   (a) incorporate java classes into an existing java gateway
   (b) how to invoke those classes from the python client side
   (c) how to get more visibility/insight into exactly what is going on /
what the meaning is of
                "Trying to call a package"

Thanks!