Ensemble

Dataset: espionage

Models

• openailong
• openai35
• random

Model Narratives

openailong

Round ID: 305
Prompt used:
	To classify an entity as either a DoubleAgent or a Loyal, follow these refined rules:
	
	1. If SecretHandshakeQuality is greater than 75 AND AccentThickness is greater than or equal to 30, classify as DoubleAgent.
	2. If SecretHandshakeQuality is between 65 and 75 AND AccentThickness is below 30, classify as DoubleAgent.  
	3. If SecretHandshakeQuality is below 65 AND AccentThickness is above 35, classify as Loyal.
	4. If SecretHandshakeQuality is between 55 and 65 AND AccentThickness is below 30, classify as DoubleAgent.
	5. If SecretHandshakeQuality is between 50 and 55 AND AccentThickness is below 30, classify as Loyal.
	6. If SecretHandshakeQuality is between 65 and 70 AND AccentThickness is between 28 and 35, classify as DoubleAgent.

Confusion Matrix:
                Predicted DoubleAgent Predicted Loyal     
Actual DoubleAgent                   10                    1
Actual Loyal                       5                    4

Accuracy: 0.700
Precision: 0.667
Recall: 0.909
F1 Score: 0.769

Examples for Correctly predicted DoubleAgent: (Correct answer: DoubleAgent, What the previous set of rules predicted: DoubleAgent)
  Entity Data:
	SecretHandshakeQuality: 67.65863
	AccentThickness: 22.4974


Examples for Falsely predicted Loyal when it should have been DoubleAgent: (Correct answer: DoubleAgent, What the previous set of rules predicted: Loyal)
  Entity Data:
	SecretHandshakeQuality: 84.02795
	AccentThickness: 23.454235


Examples for Falsely predicted DoubleAgent when it should have been Loyal: (Correct answer: Loyal, What the previous set of rules predicted: DoubleAgent)
  Entity Data:
	SecretHandshakeQuality: 63.5488
	AccentThickness: 31.045925


Examples for Correctly predicted Loyal: (Correct answer: Loyal, What the previous set of rules predicted: Loyal)
  Entity Data:
	SecretHandshakeQuality: 50.812286
	AccentThickness: 28.25855

openai35

Round ID: 467
Prompt used:
	Based on the provided examples, create a rule that considers: If SecretHandshakeQuality is above 70 and AccentThickness is below 30, predict DoubleAgent; otherwise, predict Loyal. This revised rule aims to better distinguish between the two classes based on the observed data.

Confusion Matrix:
                Predicted DoubleAgent Predicted Loyal     
Actual DoubleAgent                    7                    4
Actual Loyal                       0                    9

Accuracy: 0.800
Precision: 1.000
Recall: 0.636
F1 Score: 0.778

Examples for Correctly predicted DoubleAgent: (Correct answer: DoubleAgent, What the previous set of rules predicted: DoubleAgent)
  Entity Data:
	SecretHandshakeQuality: 81.58595
	AccentThickness: 29.12992


Examples for Falsely predicted Loyal when it should have been DoubleAgent: (Correct answer: DoubleAgent, What the previous set of rules predicted: Loyal)
  Entity Data:
	SecretHandshakeQuality: 67.65863
	AccentThickness: 22.4974


Examples for Correctly predicted Loyal: (Correct answer: Loyal, What the previous set of rules predicted: Loyal)
  Entity Data:
	SecretHandshakeQuality: 69.22898
	AccentThickness: 41.60915

random

Round ID: 138
Prompt used:
	Choose randomly

Confusion Matrix:
                Predicted DoubleAgent Predicted Loyal     
Actual DoubleAgent                    6                    5
Actual Loyal                       4                    5

Accuracy: 0.550
Precision: 0.600
Recall: 0.545
F1 Score: 0.571

Examples for Correctly predicted DoubleAgent: (Correct answer: DoubleAgent, What the previous set of rules predicted: DoubleAgent)
  Entity Data:
	SecretHandshakeQuality: 72.32254
	AccentThickness: 28.720491


Examples for Falsely predicted Loyal when it should have been DoubleAgent: (Correct answer: DoubleAgent, What the previous set of rules predicted: Loyal)
  Entity Data:
	SecretHandshakeQuality: 81.58595
	AccentThickness: 29.12992


Examples for Falsely predicted DoubleAgent when it should have been Loyal: (Correct answer: Loyal, What the previous set of rules predicted: DoubleAgent)
  Entity Data:
	SecretHandshakeQuality: 69.22898
	AccentThickness: 41.60915


Examples for Correctly predicted Loyal: (Correct answer: Loyal, What the previous set of rules predicted: Loyal)
  Entity Data:
	SecretHandshakeQuality: 50.403297
	AccentThickness: 31.491634

Ensemble Confusion Matrix

Accuracy 0.650, Precision 0.700, Recall 0.636, F1 0.667