In Maple, f:=x->fun is the definition of a function or operator. Once the operator works with (), i.e. f(x), it becomes an expression. I think your problem is that you mixed the operators with expressions. The corrected code is as follows:

The right answers are:

                             sin(x)
                             ------
                               x   
                       -cos(x) x + sin(x)
                       ------------------
                                3        
                               x         
                                                2
                3 cos(x) x - 3 sin(x) + sin(x) x 
              - ---------------------------------
                                5                
                               x                 

So it' better that operators work only with operators. Hope this will help you.