This post might be useful for maybe like 10 people who are running into issues when compiling software on Apple M* machines.

If you run into error like this:

ld: warning: ignoring file '/usr/local/lib/libpng.dylib': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/usr/local/lib/libavformat.dylib': found architecture 'x86_64', required architecture 'arm64'

And you have installed these packages using brew. I verified that these dylibs are definitely for x86_64 and the linker was not going crazy. (use lipo -archs /usr/local/lib/libavformat.dylib)

Well, that was caused because  I had installed the new M3 macbook from backup (Timemachine) and the backup was created from an x86_64.  

The fix is described here

brew bundle dump --global
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/uninstall.sh)"
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew bundle install --global

This first creates a ~/.Brewfile and then installs back those packages.