GPX trace with some extra zigzags routes, why do I get them? and how to fix it?

This is pretty common and happens particularly when inside buildings, or near to large buildings, when the GPS cannot find a reasonable set of satellites. If your device is a phone it will fall back to using cell phone data (nearest 3G or 4G masts), which will be very inaccurate, if it cannot get a correct satellite fix. Make sure you start your ride away from buildings!

After it's happened, some apps (e.g Strava) let you crop your saved route to get rid of bits like this.

Many GPSes will generate what you describe. The points that a GPS record do have error some time only 3 meters sometimes large distances. Big errors occur when reflected signal is received which will change the time that a signal takes to reach your gps and then the calculated distance from the satellite. You can improve the recording accuracy by carrying the gps higher, for example at the top of pack or high on the shoulder strap so your body isn't blocking the signals to the gps. I believe some GPSes have software that ignores or smooths silly results, they certainly do smooth their recordings otherwise even the minor zigzags would yield distances that were way over your true distance travelled.