[ GPU ] GPU Kernel creation time

[EunjuYang]

For now, it seems clCreateKernel is called whenever any type of _cl wrapper function is invoked.

For example,

FullyConnectedLayer::forwarding -> dotCl -> dot_cl -> clCreateKernel

Considering the timing of calling _cl functions, this could potentially slow down performance. (Of course, it already avoids duplicate of registration. However, it may not be enough for speed up.)
Since NNTrainer already has a compilation phase, what about to move the kernel registration process into the compilation stage?
During the compilation step, we can identify which computational units are utilized by each layer and generate the corresponding kernels accordingly.

[taos-ci]

cibot: Thank you for posting issue #2723. The person in charge will reply soon.

[jijoongmoon]

I'm thinking to change the current implementation cl context structure.
I think we need cl kernel context and cl_context has it. and when it creates cl layer, it will set the cl kernel context..
And then, it might be possible to register custom cl kernel at finalize function of cl layer.
and also Tensor kernel which is developed by us ( like gemm cl kenels ) could be initialized when kernel context is
created. (like default Kernel we provide)

[s-debadri]

PR #2732 has been created for addressing this issue. Following is the plan:

• Added registerClKernel function at cl_context to register custom OpenCl kernels as well as in-house kernels.
• Used hash map to track created Kernel objects. shared_ptr was used t store inside the map.
• Modified sscal using above Kernel creation flow to remove dependency of layer_context.

In progress: Removing layer_context dependency for existing kernels.

[EunjuYang]

[Suggestion / Need Discussion]

As I understood, current GPU's ClContext is expected to be created with LayerNode.
What about moving its first creation time to AppContext creation by adding it as a member variable to AppContext?
By doing so, we can further abstract user API to create Layer without directly calling Cl Layer's name.