Adding a Code Snippet
HTML
If you want to add a code snippet e.g. a meta tag etc to the <head>, then you should use the wp_head action:
add_action( 'wp_head', 'wpse_43672_wp_head' ); function wpse_43672_wp_head() : void { ?> <meta ..... /> <?php } There are also admin_head and wp_footer
Javascript & CSS
You could use the above code to also add Javascript, but instead custom JS should be enqueued as a JS file, or, included as inline JS via wp_add_inline_script attached to an existing enqueued JS handle.
https://developer.wordpress.org/reference/functions/wp_add_inline_script/
This is also true of CSS, which has wp_add_inline_style https://developer.wordpress.org/reference/functions/wp_add_inline_style/
Modifying the Sites Header
Note that once you do this, you will not see changes made in the parent theme when it updates for those files that you override and modify.
In A Block Theme
If your theme is a block theme then you can do this by going to the site editor and making changes in the Admin area.
In a Classic Theme
If your site is a classic PHP theme, then you can use a child theme.
To modify the header in a child theme, copy the file header.php from the parent theme into the child theme and then modify it. WordPress will see that you have a header.php in your child theme and use that instead of the parent theme header.php
Any template files you put in your child theme will take priority over the same file in the parent theme when called by WordPress.
Anything that goes in the <head> tag should be done using something such as the function in Brians answer. If it's theme specific, you can put it in a file called functions.php in your theme folder without any extra steps.