The new Console app in Sierra was a complete redesign, but also came with an entirely new logging mechanism. This changed much about the way logging works on macOS. No longer are there separate files for individual logs, but rather a Unified Logging
mechanism which centralises the logs into a single database. You can find more information about Unified Logging in the WWDC 2016 session.
Most notably for users, this change affects the way sensitive information is logged. Where the OS (or the app developer) decides that personal information is being logged, it will replace it with <private>
. This means it cannot be viewed by other apps on the system, but also means the user has no access to it, as shown in the screenshot below. Many processes such as diskarbitrationd <private>
their logs so that others cannot read the information made available through the console, avoiding leaking sensitive information.
Show private logs in macOS Catalina 10.15.3+
The following mobile configuration profile will set the required preference. This profile has been code signed and is verified on installation up to 2022.
Installing this profile will immediately make private logs visible in the Console app.
Removal of the profile can be performed through System Preferences in the Profiles.prefPane. Click the minus button in the bottom left with the profile selected to remove it and hide private logs.
Showing private logs in Catalina before 10.15.3
The private_data mode from Sierra up to Mojave appears to have been removed in Catalina, therefore the previous solution to this issue in macOS Catalina no longer works. However, not all hope is lost. Despite log telling you that private_data is an invalid mode, it's still possible to enable this.
sudo log config --mode "private_data:on"
log: Invalid Modes 'private_data:on'
Code language: JavaScript (javascript)
Saagar Jha has done some excellent research on this and discovered that the private_data mode still exists, but is prevented from being changed unless you're an Apple developer. Fortunately, a tool was released in the form of some C++ which you can find on their blog. I've compiled and code signed this code into a binary which you can download and run:
The binary has three options: status
, enable
and disable
. Run the binary without providing an option to print its usage.
$ PrivateLogs
Usage: PrivateLogs <status|enable|disable>
Code language: HTML, XML (xml)
To print the current status, whether showing the content behind <private> in logs is enabled or disabled, use status
.
$ PrivateLogs status
disabled
To show private logs, run with enable
. This must be run with sudo or as root (no error will be shown without root, but no change will occur).
$ PrivateLogs enable
$ PrivateLogs status
enabled
Showing private logs in Sierra
To show all private logs in the macOS Sierra console, run the following command in Terminal:
sudo log config --mode "private_data:on"
Code language: JavaScript (javascript)
For app developers, to override the OS’s decision on what should be made private and write publicly to the log, use the following format:
%{public}s
can you get one for Big Sur
So what about achieving the same result with external OS devices? Now that my iOS devices and my watch can natively stream their logs to my console, I would love to eliminate the same "private" masks in those logs once and for all.
Have you tried installing the profile on your iOS device?
This blog is exactly what I needed. It works on Big Sur 12.2.1! I've been having major issues with iCloud not syncing and I've been wanting to monitor it with brctl log -w, but everything was hidden with . This profile allows me to actually monitor the iCloud Syncing process now! Many thanks!
Hi, im trying to read my iOS device's logs,i tried to intsall the profile on my iOS but safari gives me the following error: "Safari could not install a profile due to an unknown error".
Does anyone knows how to fix that or know another work around to install the profile on iOS?
This worked for me with Catalina 10.15.6. Thanks!
Hi! I tried to install your magic profile, but seems like it doesn't work for me. I'm still getting console messages with
default 14:10:14.003134+0500 installd entitlement '' has value not permitted by provisioning profile ''
Does it mean that your method can't help me to understand such logs?
Hi! I tried to install your magic profile, but it doesn't work for me( I have a problem with app install and I still see only in console messages
default 13:40:49.321397+0500 installd entitlement '' has value not permitted by provisioning profile ''
Does it mean that your profile can't help me with such messages?
Do you have <private> in that message? I think the private bit has got stripped from your comment. If so, the profile should work! Does it work for other private log messages that you see (or don’t)?
Something happened with comment and private in angle brackets became ” -__-
Yes, other log messages has private keywords too for both devices (iPhone 6 with os version 12.4.4 and MacBook with 10.15.4)
I've tried to install this profile on iPhone, but catch unknown error. Can this profile help with iOS logs in console.app or its spec only Mac OS?
p.s. sorry for two similar comments, I thought first was lost
Unrelated to this specific post, but any chance you can eventually add an RSS feed for your blog? Just discovered this blog today, and it’s filled with so much useful information! I’d love to be able to keep track of new posts with my RSS feed reader.
Hi John, you can grab an RSS feed of new posts on this blog at https://georgegarside.com/feed/
Thank you very much ,It is work in 10.15.2.
Seems like this doesn't work in 10.15.3 sadly. "sudo ./PrivateLogs enable" doesn't give any errors, but "sudo ./PrivateLogs status" still shows disabled after running.
Confirm this is "fixed" in 10.15.3.
The error given (can be seen in console) is related to the binary created is not signed and not having permission to do the change. Similarly the log binary can not be inspected/changed either.
So Apple have viewed this as a security hole and plugged it as it seems
Due to additional entitlement checks, changing this setting now requires disabling System Integrity Protection and a more complicated process.
Saagar, your blog is fantastic. Thanks for that!
Have you documented the more complicated process anywhere, or do you have plans to do so?
I've updated my post with a configuration profile that enables private logging. I'd still be interested in your solution disabling SIP, feel free to leave a reply with more information.
Two questions...
Can’t seem to get the private data mode selection to STAY. I run it, restart Console, and then the next time I come back, I have to do it all over again.
Similarly, what would be the command to have the same effect on other devices that Console and Xcode can read (Watch, iOS, etc)?
sudo log config --mode "level:info, level:persist, level:debug, private_data:on"
And I don’t know why but it said there was a invalid mode but keep the party going (ignoring) because there’s more key values or something like that... I checked the console and sure enough Satan’s little Darwin’s were spilling the beans again.. Apple is sick.
You have 2 useful parameters: "level" and "stream"
and some options for each one.
"level" accepts 4 options, they can be either: off | default | info | debug
"stream" accepts 2 options, either "live" or "default"
Eg:
sudo log config --mode stream:live,level:debug
sudo log config --mode stream:live,level:info
sudo log config --mode stream:live,level:default
sudo log config --mode stream:live,level:off
sudo log config --mode stream:default,level:default
sudo log config --mode stream:default,level:info
sudo log config --mode stream:default,level:debug
There is another not-so-useful parameter called "persist"
and it accepts the same 4 options as level. This parameter is for saving the logs to the disk, and is better keep it on default.
eg:
sudo log config --mode stream:live,level:default,persist:default
You can check the status of the current parameters with:
sudo log config --status
will print some info like this:
System mode = DEFAULT STREAM_LIVE
On your command line you are trying to set "level" more than once, it will accept just one value: the last value, if multiple values are specified.
The error comes from the blank space after the comma.
and 'private_data" option does not exists anymore.
Does not work anymore since Catalina Beta2 (was working until first beta)
--
sudo log config --mode "private_data:on"
Password:
log: Invalid Modes 'private_data:on'
Urgently needing this, and no documentation at Apple.
Any clue on how to activate it now?
I looked into this and documented my progress: https://saagarjha.com/blog/2019/09/29/making-os-log-public-on-macos-catalina/. TL;DR: compile and run the code I posted at the bottom, as setting private_data mode in log is now gated by a check that I presume is meant to only be passed by Apple engineers.
Thank you very much for such detailed information linked and [for sure] for the compiled code. It is working fine on latest Catalina 10.15.3 Beta (19D49f).
But I had to execute it using 'sudo' to make it work.
Thank you!
Hi!
Newer Catalina 10.15.4 changed that, now it is officially [by Apple] possible (and is the correct way from now on..) to enable private logs is: by creating a "config profile" and installing it on the MacOs System Preferences, or on your company computers. Similar to a beta-profile, this profile has settings to enable the full log, showing all private messages.
I will post more detailed information about it here soon.
Where can I get such a profile to view the full log? Thanks!
Hi, you can get it here, below.
It works on any MacOS Catalina versions, including the just released 10.15.5 Beta (19F83c) (beta 4 from 8/may/2020)
And from now on, this is the correct way to enable viewing Private Logs. This is determined by Apple to be this way from now on.
It is easy,
1) just copy the XML below, to an empty text file, and save it as: EnablePrivateLogs.mobileconfig
2) using Finder, Double click via the file EnablePrivateLogs.mobileconfig (that you just have created)
3)Catalina will open System Preferences, and ask if are you sure that you want to install the configuration profile. It will also tell that it is unsigned and it you are sure about it. Confirm YES on those questions.
4)Profile will be installed and you can view its purpose description (which is just to manage and enable Private-Data Logs)
5) It is done, instantly you will see the private data on the Console and/or terminal Log commands
PS: A new icon will appear on System Preferences, called Profiles, and it permits you to access the profile at anytime [in case you want to remove it, just delete via System Preferences) or you can have it there forever if you want full logs forever.
Here is the XML code, copy and paste it as I described on step 1
PayloadContent
PayloadDisplayName
ManagedClient logging
PayloadEnabled
PayloadIdentifier
com.apple.logging.ManagedClient.1
PayloadType
com.apple.system.logging
PayloadUUID
ED5DE307-A5FC-434F-AD88-187677F02222
PayloadVersion
1
System
Enable-Private-Data
PayloadDescription
Enable Unified Log Private Data logging
PayloadDisplayName
Enable Unified Log Private Data
PayloadIdentifier
C510208B-AD6E-4121-A945-E397B61CACCF
PayloadRemovalDisallowed
PayloadScope
System
PayloadType
Configuration
PayloadUUID
D30C25BD-E0C1-44C8-830A-964F27DAD4BA
PayloadVersion
1
code formating test
[code] code test using BB tag [/code]
code format via html pre tag
(Sorry for this, I i trying to paste a formatted code here, please delete this test message later)
Thanks
Thanks for this, I’ll update the post with the XML tomorrow. I think I can guess what the tags are meant to be, looks like they all got stripped as bad HTML. At some point I need to get Markdown working in comments 🙂