When we write a program say
var a = array
var b = array
var c = array
a = random()
b = random()
c = a +b
We think as CPU run the code.
But in TVM or TensorFlow or ComputeGraph
what we do is not create a program that run as it is.
what we do is create a description a function
and then we apply some constrain on the function.
like data type
like how to move data
like how to compute data
then we generate the Real Code of the function.
Because in the old day, we have one Memory one CPU
Now we have different CPU with different memory structure
the Simple mind programming model is not working anymore
it is hard for us to think or reason in different computation model
before TPU , Compiler help us to maintain the old thinking.
We need to think totally different.
We rescript computation and let computer figure it out
how to apply on different CPU/GPU/FPGA/TPU