I work with Powershell quite often. I make healthcheck systems for apps such as NetBackup, and others. I use it for day to day administration of NetApp and vmware.
One problem I always have with it jumping between customer environments is the fact that the default text output is UTF-16. This looks like a bunch of junk when I view it in terminal on my Mac, and also it is not easily grep’able, or less/more’able.
What a more against the file looks like:
|
1 2 3 4 5 |
jk47s-laptop:client_images jkulm$ more m041022.txt "m041022.txt" may be a binary file. See it anyway? <FF><FE>C^@L^@A^@S^@S^@ ^@m^@s^@_^@t^@w^@i^@n^@_^@b^@ ^@*^@N^@U^@L^@L^@*^@ ^@0^@ ^@7^@1^@0^@3^@0^@0^@ ^@0^@ ^@*^@N^@U^@L^@L^@*^@^M^@ ^@N^@A^@M^@E^@S^@^M^@ ^@I^@N^@F^@O^@ ^@1^@3^@ ^@0^@ ^@0^@ ^@2^@ ^@*^@N^@U^@L^@L^@*^@ ^@0^@ ^@0 |
Why? Well thats how the terminal converts the unicode.
Typically in Powershell, you can issue this command, and force the Unicode output to be ascii:
|
1 |
$OutputEncoding = New-Object -typename System.Text.ASCIIEncoding |
The problem is, it doesn’t always work. I haven’t looked too far into the WHY of it not working, but it’s a big ol pain in the ass and when I need it to, it doesn’t.
Cleaning up this text is INCREDIBLY easy! 1 command. That’s it. Iconv to the rescue. You can use portage to install it. “sudo port install iconv”
Make a file in your users ~/bin/ directory called catutf16
|
1 2 |
#!/bin/bash iconv -f UTF-16 -t UTF-8 $1 |
Then change the perms to make it executable
|
1 |
chmod u+x ~/bin/catutf16 |
After that, you can simply process ALL files or just one file at a time to output it to a new file in the right text encoding.
Want to see one file?
|
1 2 3 4 5 6 |
jk47s-laptop:client_images jkulm$ catutf16 m041022.txt | head Backed Up Expires Files KB C Sched Type Policy ---------------- ---------- -------- -------- - ------------ ------------ 05/10/2012 23:12 05/12/2012 252 11350 N Cumulative I 3_unix_twin_b 05/09/2012 23:21 05/11/2012 226 11254 N Cumulative I 3_unix_twin_b 05/08/2012 23:08 05/22/2012 222 11158 N Cumulative I 3_unix_twin_b |
Want to convert ALL files in a directory?
|
1 2 |
jk47s-laptop:client_images jkulm$ mkdir utf8 jk47s-laptop:client_images jkulm$ for i in *.txt; do ~/bin/catutf16 $i > utf8/$i; done |
It’s just that easy. UTF-16 to UTF8. You can change any of the charactersets as your destination.
