The following command get the objects related to a file group:

SELECT OBJECT_NAME(object_id), *
FROM sys.data_spaces ds
INNER JOIN sys.allocation_units au ON
	ds.data_space_id = au.data_space_id
INNER JOIN sys.partitions p ON
	au.container_id = 
		CASE 
			WHEN au.type = 2 THEN p.partition_id
		ELSE p.hobt_id 
		END
WHERE ds.name = 'MyFileGroup'

This gives one record. In that record, there is not really any user friendly name of which file it is. Anyone would know, from there, what to do to get the file so I can remove the file first and then retry this command:

ALTER DATABASE MyDatabase REMOVE FILEGROUP [MyFileGroup]