Improve git ignore handling

Do not add the same thing over and over again to reduce output on the CLI.