I doubt there is a stable terminology here. A small googling exposes people utilize base name for this, but this usage conflicts with the same term used as "full file name without path". For example, https://stackoverflow.com/questions/2183486/ exposes the case when "filename" is "basename" plus "extension".
OTOH, the term "extension" you use is weird. It's originated in CP-M/Dos/Windows world (distorting imitation of RSX-11/RT-11 approach) where a file name could (before Windows 95) have a single part after dot. In Unix world, this was being named "suffix" from the very beginning, and I would strongly suggest you using this term. The difference is that multiple suffixes can exist - for example, a.o.d is dependency list for a.o. In turn, if this is suffix, the part before suffixes is a filename root. In my opinion, there is too small chance to get it conflicting with "root" as Unix superuser.
UPD: linguistics often use "stem" in a vast sense "part before ending and, sometimes, before suffix". But I see mention for "root" in neighbor reply, so, seems "root" is good until you need further distinction.