You can add it to registry with the following code
My.Computer.Registry.LocalMachine.OpenSubKey("SOFTWARE\Microsoft\Windows\CurrentVersion\Run", True).SetValue(Application.ProductName, Application.ExecutablePath)
you can remove it using
My.Computer.Registry.LocalMachine.OpenSubKey("SOFTWARE\Microsoft\Windows\CurrentVersion\Run", True).DeleteValue(Application.ProductName)
The above code will add it to all users. You can add it to current user in the following key
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
Or you can add a link to your application in the "Startup" folder.
I suggest you dont do it automatically it may irritate the user. I hate when apps add them automatically to Windows Startup. Give an option for the user to run the program on windows startup.