I wanted to make an embedded script in powershell and stumble upon your same problem.
The thing is that Microsoft.SqlServer.SqlClrProvider.dll is not installed in SQL files but in windows assembly.
Assembly folder is special: you can't copy anything with windows UI.
I just needed the Microsoft.SqlServer.SqlClrProvider.dll so that I can load it in my script like that:
[System.Reflection.Assembly]::LoadFile($scriptPath+"dll\Microsoft.SqlServer.SqlClrProvider.dll") | out-null
to copy it you can do the following:
c:\>cd c:\Windows\assembly
c:\Windows\assembly>dir /s Microsoft.SqlServer.SqlClrProvider.dll
# here you get the directory and the file
c:\Windows\assembly>copy c:\Windows\assembly\GAC_MSIL\Microsoft.SqlServer.SqlClrProvider\<version>\Microsoft.SqlServer.SqlClrProvider.dll c:\Users\Administrator\Desktop\
you want to do that on a machine with SQLserver installed otherwise you may not find it.
It's just a workaround. I guess it's not the best thing to do if you need to execute your script in different environments.
hth