Move database file location non existing file

When the database is moved from another location, attach the database as:
CREATE DATABASE [test] ON (
( FILENAME = N'N:\Data\test.mdf' ), FILENAME = N'N:\Log\test.LDF' )
GO

There is a secondary filegroup for this database. The file location is same as orginal file location. We cannot update or remove this secondary file:
ALTER DATABASE testMODIFY FILE(NAME = test_secondary,FILENAME = N'N:\data\test_secondary.ndf');

It will fail because the file group is pending recovery.

ALTER
DATABASE test SET OFFLINEFOR ATTACH