I've seen benchmarks (eg: this one) that put btrfs considerably slower than ext4. The benchmark I linked attributes this to copy-on-write behaviour of btrfs.
However benchmarks test quite narrow parameters which may not be reflected by running an OS. For example btrfs supports transparent file compression. This may speed up the otherwise slower file system considerably at the expense of CPU.
A big proportion of OS IO is based on small file reads. This means that slow down due to reading data on disk is much less than you might expect when compared to spinning hard drives. That's because USB flash drives have very low "seek time", running Ubuntu from a flash drive is pretty smooth.
As for wearing out a flash drive, this is more of a problem. If you use it heavily then you will eventually wear out a USB drive where you might not ever wear out of a hard drive. F2FS is your better option (Yes I realise it wasn't in your question). This was designed to reduce flash wear and importantly designed to spread flash wear.
Quite often you don't need to worry though... It will depend on you and how much money you have. Think about whether or not you really care about spending another $10 in a year or two.
A lot of the better usb flash drives (well known brands with a reputation to maintain) will fail into read-only mode. Ext4 is designed to be very hard to corrupt so when the time comes you're unlikely to lose data if you formatted with ext4.
If you are still concerned about wear then you might want to do something a little funky with your setup: make the windows PC hard-drive host part of your file-system.
You can create a large single file on your main Windows hard drive and then use it as a loopback device, (that file can be used like a hard drive). This will let you put frequently changing areas of your file system (/var and /home) onto your Windows PC and keep them away from your USB drive.
However this approach will mean your pen-drive linux will no-longer boot on any machine, only yours.