I tried this but it doesn't work:

    ' Stop the service
    Private Function StopService() As Boolean
        Dim loServiceController As System.ServiceProcess.ServiceController = New System.ServiceProcess.ServiceController()

        loServiceController.MachineName = "."
        loServiceController.ServiceName = "MyService"

        ' Stop the service
        loServiceController.Stop()

        Return True
    End Function

Basically, MachineName represents the PC itself. Then, ServiceName is the exact service name as defined in my project. However, the Stop() doesn't generate an error in the event viewer but the service still runs.